Learning the Terminal

Mon, Aug 29, 2016

Readings

Assignment

Complete the in-class tutorials on your own.

Understanding the Command Line Interface

The Terminal program on the Mac is a window into the command-line interface (CLI). This is the ability to interface with your computer using text commands, much the way it was done on Unix/Linux computers.

Terminal window

The command-line interface is also known as the Terminal, Shell or Console. The Mac Terminal is actually a program called Bash, which is it is sometimes referred to (or a combination of words, like Bash shell).

Listing Files in the Terminal

The first command we will use is the ls command, which stands for list, as in “list the current directory.”” When you launch Terminal, you generally start out in your home folder.

~$ ls

Listing in the terminal window

After typing in the ls command, you will see the folders that are in your home directory listed.

The ls command can also be used with certain options called flags, typically with a dash prefix. For example, there is also the ls -a command which not only displays the main files, but any hidden files. Hidden files typically begin with a dot . which make them hidden on a Mac.

~$ ls -a

Listing hidden files in the terminal window

In this example, we see a lot of extra hidden files and folders we didn’t see before. (Your computer will likely show different files and folders that above.)

There is another flag -l which will present the ls command as a long list with lots of extra data about the files, such as the permissions and size of each file. We can also combine both the -a and -l like so ls -al

~$ ls -l

drwxr-xr-x    3 admin  staff    102 Jul  4  2013 Applications
drwx------+   0 admin  staff   1394 Nov 11 22:24 Desktop
drwx------+  15 admin  staff    510 Aug  2 16:00 Documents
drwx------+  86 admin  staff   2924 Nov  2 16:06 Downloads
drwx------@  63 admin  staff   2142 Aug  2 16:16 Library
drwx------+   6 admin  staff    204 Aug 26 16:00 Movies
drwx------+   6 admin  staff    204 Aug 16 16:00 Music
drwx------+   4 admin  staff    272 Sep 18 14:05 Pictures
drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public
drwxr-xr-x    0 admin  staff    138 Sep  3 17:00 Sites

Linux File Permission Notation

Let’s take one of the lines from the long list above and de-construct each part of it:

File Type

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The very first letter “d” (shown in red above) refers to what type of file it is.

  • `d` means this is a directory (folder)
  • `l` means this is a link, also known as a symbolic link, or alias
  • `-` means this is a normal file

Permissions

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The next part specifies the permissions of this file, and who can access it. It is comprised of three letters R-W-X. It’s the part shown in red below:

  • r stands for read permissions
  • w stands for write permissions
  • x stands for execute permissions

Permission diagram

  • Owner typically refers to the user who made the file (the one logged in).
  • Group refers to a certain class of users arranged into a group, typically in enterprise environments (or anyone else you want to grant access to).
  • World/Everyone is anyone else who has the ability to access this folder or file, will have these permissions.

Here are a few examples:

  • -rwx------ The owner can read, write or execute. But the group and world have no access to this file.
  • -r-xr-xr-x Owner, group and world can all read the file, and execute it, but not change it in any way.
  • -rwxr----- Owner can read, write and execute, while the group can only read the file.

Specifying Permissions as Numbers (Octal)

Permissions can also be represented by numbers. This is called octal notation.

  • 0 no permission, same as a dash
  • 1 execute only
  • 2 write only
  • 3 write and execute (1+2)
  • 4 read only
  • 5 read and execute (4+1)
  • 6 read and write (4+2)
  • 7 read, write and execute (4+2+1)

Each class of permission (owner, group, world) is represented by one number. In combination, these three numbers can translate the letters R W and X for letter permissions.

Symbolic Notation Octal Notation English
---------- 000 no permissions
---x--x--x 111 execute
--w--w--w- 222 write
--wx-wx-wx 333 write & execute
-r--r--r-- 444 read
-r-xr-xr-x 555 read & execute
-rw-rw-rw- 666 read & write
-rwxrwxrwx 777 read, write, & execute

Extended permissions

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The last symbol, which only shows up sometimes as a + or @ refers to special extended permissions, usually applied by the Mac operating system.

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The next number specifies how many other folders or linked files are inside this directory.

Owner and Group names

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The next part specifies who owns this file (admin), and what group (staff) also may have access to this file/folder, depending on the permission set. In the permissions segment drwxr-wr-x, the rwx refers to admin, since admin is the owner, while the r-w refers to the staff group.

File size and date

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

The next segments show you the size of this file in bytes, and the date it was modified.

Name of file or folder

drwxr-xr-x+   4 admin  staff    136 Jun  8  2013 Public

Typing the cd command, means to change directory.

~ $ cd Desktop

The dot . means the current directory.

The double dot .. means the parent directory.

The tilde ~ always points to your home folder.

Desktop $ cd ~

The above example would change directory to the home folder regardless of where you are.

Regular Expressions (RegEx)

Regular Expressions (RegEx for short) is a way to search for strings — characters like words, numbers or symbols — within a document. It uses special syntax which matches various types of characters.

In a typical search function you can look for a particular word in your document. But, what if you wanted to do a more sophisticated search? For example, maybe you wanted to find words followed by a number, and only appearing at the end of a sentence. Or maybe you want to search a database for e-mail addresses, or specially formatted phone numbers. These would be impossible to do with typical search functions; RegEx to the rescue!

There are different ways for finding strings in a document. Let’s start by using a program like Sublime Text 3.

Open a new window, and then pressing Command F. This will open a special search bar.

Sublime search bar

Make sure to click the button at the left signified by a .* to turn on RegEx mode. This will ensure that all searches are interpreted by RegEx.

Dummy Data

Below is some dummy data we’ll use to test out our RegEx code. It is a .csv file, so it follows a standard structure that we can use for searching and replacing.

Copy and paste the data below into Sublime Text.

Names,Email,Address,City,Zip
Anthony Velazquez,Sed.eu.nibh@orci.co.uk,"815-9466 Id, Rd.",Sint-Amandsberg,9076
Nelle Melton,lobortis.Class@acfacilisisfacilisis.org,3310 Enim Road,Lidköping,441916
Imelda Eaton,tincidunt@Crasvulputate.org,934-2547 Sit Rd.,Maidenhead,4605KG
Mariko Gill,Quisque@arcuSedeu.org,Ap #910-3385 Adipiscing St.,Cádiz,9654
Halla Stone,euismod.urna@nislMaecenas.co.uk,"P.O. Box 123, 2605 Enim Street",Noduwez,84744
Imogene Osborne,condimentum.eget@iaculisnec.co.uk,"4923 A, Street",Curanilahue,46232
Blythe Andrews,et@Classaptenttaciti.net,"P.O. Box 778, 9229 Elit, Road",Palanzano,66938
Althea Coleman,id.ante.Nunc@ornare.edu,"P.O. Box 251, 7623 Sem Rd.",San Vicente,71147
Aurora York,metus@Proineget.ca,3080 Eu Ave,Monte Santa Maria Tiberina,616988
Bert Greene,nec@idrisus.com,Ap #104-8518 Mauris Av.,Tullibody,03829
Hilary Anthony,faucibus@est.co.uk,977-8129 Interdum. Ave,Denbigh,748074
Oleg Osborn,est.congue.a@dis.co.uk,5903 Libero St.,Lloydminster,826419
Reese Carson,tempor.erat@eratvolutpatNulla.com,"P.O. Box 986, 8476 Sed St.",Port Hope,56311
Desiree Bender,sollicitudin.commodo.ipsum@volutpat.org,"P.O. Box 709, 9409 Pede Rd.",Casole d'Elsa,28970
Aileen Ruiz,libero.at.auctor@eterosProin.org,Ap #331-6282 Augue Rd.,Saint-Pierre,59430
Shoshana Crawford,erat.volutpat.Nulla@volutpat.com,7112 Ligula. Av.,Cavallino,766727
Berk Morrow,tristique.senectus@mauris.ca,"P.O. Box 455, 2971 Non, St.",Namur,25-237
Karina Burns,tincidunt.Donec@utcursus.net,"8333 Nec, Av.",Cape Breton Island,79-736
Yael Hartman,molestie@bibendumullamcorper.edu,256 Lacus. Av.,Augusta,101472
Neil Collins,porttitor.interdum@tellus.ca,"Ap #153-8862 Consequat, St.",Fauglia,2781
Vernon Lindsay,Donec@interdumligulaeu.net,Ap #935-6426 Elit Street,Braies/Prags,941743
Daria Klein,velit.Quisque.varius@felis.net,106-8489 A Av.,Evansville,15-001
Medge Lara,Morbi@mattisornare.org,1484 Aliquam Rd.,Napoli,71765-492
Aladdin Douglas,aliquam@commodo.co.uk,2649 Nullam Street,Hyderabad,11-038
Hayes Carlson,imperdiet.dictum@nisiCumsociis.com,"741-6653 Quis, Ave",Newport,22415
Jared Pickett,semper.erat@necluctus.edu,8195 Fusce Rd.,Ribeirão Preto,49751
Jonah Odom,metus.Aliquam.erat@odio.co.uk,Ap #520-4546 Pellentesque Av.,Pietrarubbia,08329-197
Rama Leonard,dolor.Quisque@egestas.co.uk,8080 Cras St.,Sankt Ingbert,3125
Charles Kemp,viverra.Donec.tempus@morbitristique.org,"P.O. Box 973, 9604 A, Ave",Livingston,9168
Aimee Winters,porttitor.scelerisque@velitPellentesqueultricies.co.uk,Ap #191-9756 Fusce Road,Sauvenire,G4 6QE
Norman Buchanan,vel.est@euismodacfermentum.co.uk,"P.O. Box 649, 6495 Non Avenue",Ried im Innkreis,51109
Lilah Marshall,tortor@a.co.uk,"P.O. Box 101, 468 Est Street",Bad Vilbel,2012

As the following comic implies, RegEx can be a very tricky system to master and often results in lots of errors, spurious results, or invalid data. Use with caution. Still, it’s an important system to know about.

Follow the Regex One Interactive Tutorial.

XKCD comic