Chris Conlan

Financial Data Scientist

  • About
  • Blog
    • Business Management
    • Programming with Python
    • Programming with R
    • Automated Trading
    • 3D Technology and Virtual Reality
  • Books
    • The Financial Data Playbook
    • Fast Python
    • Algorithmic Trading with Python
    • The Blender Python API
    • Automated Trading with R
  • Snippets

Printing My Own Newspaper: Building Our Own Tools

December 14, 2016 By Chris Conlan 1 Comment

Overview

Bias in news media was a point of contention in the 2016 election season. Everyone is familiar with this topic, so I will not attempt to explain further or give any more background.  I did what any reasonable statistician would do to reduce bias: I took a random sample. More specifically, I used the R language to scrape the top 10 headlines from a handful of news outlets and displayed them in a random order. The end result is an HTML page with headlines and links to the original articles.

my_newspaper

There are many skills required to write software like this.

The goal of this article is not to teach all of these skills, but to remind developers to build tools for themselves.

To illustrate why it felt very natural to him to build the Linux Kernel, Linus Torvalds explains:

I got used to the fact that you couldn’t even buy ready-made programs… so I wrote my own tools for this and that.

Link to Linus Torvalds Talk

He goes on to talk about how he built sloppy games and text editors out of necessity all of the time, simply because he couldn’t find or couldn’t afford what he needed.

Now That’s How You Use a Computer

The newspaper tool discussed here is a very sloppy and quick tool for my personal use. It is of little consequence if it breaks now and then, or even completely fails to run. There is a line in my crontab file that tells it to run every day at 6:00 a.m. and 6:00 p.m. when I am most likely to check the news.

My /etc/crontab file (manages task scheduling in UNIX) is packed with over a dozen lines that automate periodic tasks. The jobs handled by my crontab include:

  • Periodically updating OS-critical processes
  • Running custom spam filters on various email accounts
  • Creating git-powered progress and work reports on developers and contractors
  • Creating long-term system usage reports
  • Identifying malicious network traffic, IP-blocking it, and generating reports on it
  • Updating predictive models
  • Creating my personal newspaper

Every developer has different skill sets that enable him to improve his life in specific ways.  With a little bit of creativity, most full-featured languages can accomplish most of the things listed above. I encourage this generation of developers to not forget how earlier ones were unhesitating in their desire to build custom personal software tools.

Bonus: My Newspaper’s Software Stack + Source Code

  1. The R Language
  2. R packages knitr, rmarkdown, RCurl, httr, and XML with dependencies.
  3. Any web browser.
  4. CRON for task scheduling in UNIX, or Task Scheduler for task scheduling in Windows.

Download Newspaper Generator for UNIX environments, pre-configured to work out of the directory ~/Desktop/newspaper/ by executing run.sh.

Filed Under: Chris Conlan Blog, Programming with R Tagged With: Building your own newspaper, Creating tools that work, developing tools that matter online Chris Conlan, Print Newspaper with R

Comments

  1. Kim says

    December 15, 2016 at 10:42 pm

    We should talk about the Newspaper wold. Like your insight. Thanks for posting.

    Reply

Leave a Reply Cancel reply

Latest Release: The Financial Data Playbook

The Financial Data Playbook

Available for purchase at Amazon.com.

Algorithmic Trading

Pulling All Sorts of Financial Data in Python [Updated for 2021]

Calculating Triple Barrier Labels from Advances in Financial Machine Learning

Calculating Financial Performance Metrics in Pandas

Topics

  • 3D Technology and Virtual Reality (8)
  • Automated Trading (9)
  • Business Management (9)
  • Chris Conlan Blog (5)
  • Computer Vision (2)
  • Programming with Python (16)
  • Programming with R (6)
  • Snippets (8)
  • Email
  • LinkedIn
  • RSS
  • YouTube

Copyright © 2022 · Enterprise Pro Theme On Log in