Printing My Own Newspaper: Building Our Own Tools

Overview

Bias in news media was a point of contention in the 2016 election season. Everyone is familiar with this topic, so I will not attempt to explain further or give any more background.  I did what any reasonable statistician would do to reduce bias: I took a random sample. More specifically, I used the R language to scrape the top 10 headlines from a handful of news outlets and displayed them in a random order. The end result is an HTML page with headlines and links to the original articles.

There are many skills required to write software like this.

The goal of this article is not to teach all of these skills, but to remind developers to build tools for themselves.

To illustrate why it felt very natural to him to build the Linux Kernel, Linus Torvalds explains:

I got used to the fact that you couldn’t even buy ready-made programs… so I wrote my own tools for this and that.

Link to Linus Torvalds Talk

He goes on to talk about how he built sloppy games and text editors out of necessity all of the time, simply because he couldn’t find or couldn’t afford what he needed.

Now That’s How You Use a Computer

The newspaper tool discussed here is a very sloppy and quick tool for my personal use. It is of little consequence if it breaks now and then, or even completely fails to run. There is a line in my crontab file that tells it to run every day at 6:00 a.m. and 6:00 p.m. when I am most likely to check the news.

My /etc/crontab file (manages task scheduling in UNIX) is packed with over a dozen lines that automate periodic tasks. The jobs handled by my crontab include:

• Periodically updating OS-critical processes
• Running custom spam filters on various email accounts
• Creating git-powered progress and work reports on developers and contractors
• Creating long-term system usage reports
• Identifying malicious network traffic, IP-blocking it, and generating reports on it
• Updating predictive models
• Creating my personal newspaper

Every developer has different skill sets that enable him to improve his life in specific ways.  With a little bit of creativity, most full-featured languages can accomplish most of the things listed above. I encourage this generation of developers to not forget how earlier ones were unhesitating in their desire to build custom personal software tools.

Bonus: My Newspaper’s Software Stack + Source Code

1. The R Language
2. R packages knitr, rmarkdown, RCurl, httr, and XML with dependencies.
3. Any web browser.
4. CRON for task scheduling in UNIX, or Task Scheduler for task scheduling in Windows.

Download Newspaper Generator for UNIX environments, pre-configured to work out of the directory ~/Desktop/newspaper/ by executing run.sh.