notes

Part 1 of notes (sorry, this will be made neater soon)

1. Uploading your files to your very own filehost!
2. The Clam and the Spider (the shell and the web)
- 2.1. youtube-dl
- 2.2. scraping
  - 2.2.1. jq
  - 2.2.2. html-xml-utils
3. atool
4. ranger
5. ssh config
6. readline vi mode
7. dotfiles repo
8. script, scriptreplay
9. OpenAFS
10. email
- 10.1. show my use of notmuch and a shell loop to grab PGP keys from emails
11. sed
12. w3m -dump lel
13. todo.sh
14. gnuplot
15. learning about unix utilities
16. searching files

1 Uploading your files to your very own filehost!

rsync $FILE unix.andrew.cmu.edu:~/www/
firefox www.contibu.andrew.cmu.edu/~sbaugh/$FILE

2 The Clam and the Spider (the shell and the web)

2.1 youtube-dl

Ha ha so neat! Also works with sites like Bandcamp, Vimeo, and others!

youtube-dl https://www.youtube.com/watch?v=LYYUcvPkxtg

2.2 scraping

2.2.1 jq

This is "sed for JSON", a way to process JSON from the shell. curl | jq

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq .
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq .[0]
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq .[0].commit.author

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '[.[] | {message: .commit.message, name: .commit.committer.name}]'

2.2.2 html-xml-utils

This is a collection of useful utilities for simple web scraping from the shell. I use hxwls the most. curl | hxwls

curl http://www.reddit.com/ | hxwls | grep imgur | sort | uniq | grep http | xargs wget

3 atool

Much easier than manually using tar! Also has als!

4 ranger

Useful for exploring directory trees!

5 ssh config

Shared, persistent connections!

# ~/.ssh/config
Host *
    # automatically share a connection through a socket
    ControlMaster auto
    # create the socket at a global path
    ControlPath /tmp/%r@%h:%p
    # leave the connection open after the last shell closes
    ControlPersist 10m

6 readline vi mode

# ~/.inputrc
set editing-mode vi
set show-mode-in-prompt

7 dotfiles repo

Homeshick has worked well for me.

8 script, scriptreplay

I have to admit, I've never actually used this. But it's still neat!

script --timing=times content
scriptreply times content

9 OpenAFS

Handy, if you're running Linux and therefore it's easy for you to get it set up

10 email

offlineimap notmuch msmtp alot

10.1 show my use of notmuch and a shell loop to grab PGP keys from emails

C-r "notmuch"

11 sed

sed (if you're scared of learning it, keep in mind that it is identical to the find and replace you're used to in vim! easy!)

12 w3m -dump lel

w3m -dump cmucc.org | tr -s '[:blank:]' '\n' | sort | uniq -c
links

13 todo.sh

14 gnuplot

seq 5 | awk '{print 2*$1, $1*$1}' | feedgnuplot --lines --points --legend 0 "data 0" --title "Test plot" --y2 1 --terminal 'dumb 80,40' --exit

15 learning about unix utilities

coreutils man pages util-linux man pages Googling for things!

15.1 examples

cal date mktemp pgrep cloc

15.2 shell builtins

disown

15.3 linux

lsblk sudoedit

15.4 networking

dig arp???

16 searching files

ag, grep, git grep