Data Science at the Command Line

Obtain, scrub, explore, and model data with unix power tools.

Introduction

The unix command line, although invented decades ago, is an amazing environment for efficiently performing tedious but essential data science tasks. By combining small, powerful, command-line tools (like parallel, jq, and csvkit), you can quickly scrub and explore your data and hack together prototypes.

This hands-on workshop is based on the O’Reilly book Data Science at the Command Line, written by our CEO Jeroen Janssens. You’ll learn how to build fast data pipelines, how to leverage R and Python at the command line, and how to quickly visualise data. No prior knowledge about the unix command line is required.

By the end of this workshop you will have a solid understanding of how to integrate the command line in your data science workflow. Even if you’re already comfortable processing data with, for example, R or Python, being able to also leverage the power of the command line can make you a more effective and efficient data scientist.

What you’ll learn

  • Automate tedious tasks
  • Parallelise and distribute your tasks to multiple cores and machines
  • Convert your existing code to reusable command-line tools
  • Easily inspect, transform, and visualise data
  • Apply a variety of supervised and unsupervised machine learning algorithms

Schedule

Day 1:

  • Introduction
    • What is the command line?
    • Why learn the command line for doing data science?
    • A real-world data science use case
    • Getting up and running with the Docker image
  • Essential concepts of the unix command line
    • Running command-line tools
    • Combining command-line tools
    • Redirecting input and output
    • Working with files
    • Getting help
  • Obtaining data from logs, spreadsheets, and databases
  • Downloading data from the Internet and accessing APIs using curl
  • Transforming data with filters such as cut, paste, grep, and sed
  • Processing other data formats efficiently
    • JSON with jq
    • CSV with csvkit
    • HTML with pup
    • XML with xmlstarlet

Day 2:

  • Running R from the command line
  • Visualising data from the command line
    • Scatter plot
    • Histogram
    • Bar chart
    • Geographic visualisation
  • Parallelising and distributing data-intensive pipelines
  • Creating reusable command-line tools
    • Automate things in a Bash script
    • Convert your existing code to a command-line tool
    • Processing arguments
    • Working with streaming data
  • Applying machine learning
    • Outlier detection
    • Dimensionality reduction
    • Classification
    • Regression
  • Conclusion

Participants are kindly requested to have the following items installed prior to the start of the workshop:

  • Docker Desktop for Windows or for Mac or for Ubuntu
  • The docker image, by running: docker pull datascienceworkshops/data-science-at-the-command-line

About your instructor

Jeroen Janssens
Principal Instructor, Data Science Workshops

Jeroen is an RStudio Certified Instructor who enjoys visualizing data, building machine learning models, and automating things using either Python, R, or Bash. Previously, he was an assistant professor at Jheronimus Academy of Data Science and a data scientist at Elsevier in Amsterdam and various startups in New York City. He is the author of Data Science at the Command Line. Jeroen holds a PhD in machine learning from Tilburg University and an MSc in artificial intelligence from Maastricht University.

Clients

We’ve previously delivered this workshop at:

Teradata
SURFnet
Snow
Amazon
Accenture
Social Point
Container Solutions
The New York Times
Prezi

Photos and testimonials

Sanne Bouwman
Data Scientist, Teradata

Great workshop! Very well done and very useful information delivered in an excellent and interactive manner. Jeroen anticipated very well on the different knowledge levels within the group. I would highly recommend the Data Science at the Command Line workshop to anyone that is interested in either kickstarting their command-line experiences or improving their data science with Unix power tools.

Joost van Dijk
Manager Middleware Services, SURFnet

As a seasoned UNIX command line adept, I didn’t expect to learn much from a Data Science at the Command Line workshop. I was wrong! Over the years, many new tools have become available that I didn’t know about, and that can be combined with traditional tools in new ways.

Since attending the workshop, I have been able to simplify and improve the efficiency of many of the scripts I use on a daily basis. Recommended for anyone working from the command line, newbies and ninjas alike!

Marc Canaleta

Besides demonstrating a good knowledge and experience in command-line tools for data science, the instructor had very good training skills, clear communication, and managed to adapt the level of the training to the level of the audience, which is not always easy!

Sign up

One upcoming date:
We can also organise this hands-on workshop as an online training for your team. Learn more.
One-day hands-on workshop
Introduction to Linux and the Bash Shell
Course description unavailable. Please contact us to learn more.
€ 449 per person
One-day hands-on workshop
Mastering Version Control with Git and Github
Course description unavailable. Please contact us to learn more.
€ 399 per person