Day 3: Tuesday, January 17, 2017 - Practical Pipes¶
- Login to an iMac
- Open the Terminal
- Run this command, which will install the
soxsoftware on the iMac
brew install sox libav.
- Then, follow the instructions on installing some software tools on your Stanford FarmShare space: Installing software specific for Compciv 2017
This is the last week that we focus specifically on the Nix command-line environment. It’s not so much as memorizing the commands/intricacies of the Unix-like shell, but rather, understanding how programs – of any language – work together, particularly through the interface of text.
For Tuesday, we’ll focus on the learning by doing. On Thursday, we’ll take a slightly deeper dive into the concepts before we slowly dip into the Python programming language.
Take a quick look at this guide, which I will be fleshing out on a need-to-know basis: The abridged guide to shell scripting and the command line
Just a quickie data-text problem
Connecting programs through text¶
Big programs are just small programs neatly tied together. And it’s a natural consequence of following the Unix Philosophy:
This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
Though this is not a class about Unix/Bash programming, designing programs to be simple and to communicate through text are the principles we will carry into Python programming, and computing in general.
Computers are dumb and simple. So we make our programs dumb and simple.
Writing shell scripts¶
From that same Bell Labs video, programmer Lorinda Cherry discussing how Kernighan’s example of a spellcheck can be written as a script (starts at 15:37).
Understanding structured text as data¶
Regular expressions and
grep is so much fun, why not use it for all kinds of data parsing?
The csvkit suite of tools¶
No more parsing structured text through regular expressions.
One of our projects down the road is to write a bot: A remote bot
In honor of this week’s Inauguration Day, let’s look at bots built around our new President, or specifically, his Twitter account.
In the wake of Trump’s electoral upset, a lot of words have been written on the bots that purportedly boosted him:
- Automated Pro-Trump Bots Overwhelmed Pro-Clinton Messages, Researchers Say: https://www.nytimes.com/2016/11/18/technology/automated-pro-trump-bots-overwhelmed-pro-clinton-messages-researchers-say.html
- On Twitter, Trump bots are out-tweeting Clinton bots 7 to 1: http://www.recode.net/2016/11/1/13488020/trump-bots-clinton-twitter-third-debate-twitterbots-election
- Inside Donald Trump’s Twitter-Bot Fan Club <http://nymag.com/selectall/2016/06/inside-donald-trumps-twitter-bot-fan-club.html
But let’s look beyond politics and popularity. Let’s look at money:
The “Every Trump Donor” (@EveryTrumpDonor) account reads through FEC data to create automated mini-stories/facts, not much different than what @everyword did for dictionaries:
via The Atlantic: The First Reply to a Trump Tweet Is Prime Media Space:
Notching the first-reply spot for any Trump tweet—let alone several of them—takes a combination of super-speed and luck. You’re up against countless people and bots—and the bots are programmed to reply automatically to Trump as a way of leveraging his reach.
This bot activity includes an entire cottage industry of internet-designed T-shirts and mugs, with bots advertising slogans like “enjoying my hot cup of liberal tears,” all within replies to Trump tweets.
Here’s an example Tweet purportedly targeted at people who read Trump’s tweets:
via Motherboard/VICE, Twitter has made some attempt to shut down these spambots: Twitter Has Suspended the TrumpBots
But there’s real money to be made beyond coffee mugs:
- This App Wants to Help You Trade Stocks Based on Trump Tweets
- Traders scheme to cash in on Trump tweets
- Trump’s tweet about Lockheed-Martin cuts $4bn in value as share prices fall
The F-35 program and cost is out of control. Billions of dollars can and will be saved on military (and other) purchases after January 20th.— Donald J. Trump (@realDonaldTrump) December 12, 2016
Thoughts on tracking the Trump Administration¶
Bots are fun to program. But there are many other (mostly manual and mundane) ways to track the incoming Trump Administration, and more useful ways to display and present the information than via the tweets of a bot (i.e. a web application, which is the final project)
This TrumpTracker project is a built off of a spreadsheet of collected promises: https://trumptracker.github.io/
BuzzFeed is making a spreadsheet of connections: Help Us Map TrumpWorld
The Washington Post has a fantastic list of all the nominations made and the ones to be made: Tracking how many key positions Trump has filled so far
Some examples from the other side of the aisle: