Introduction
The unix command line, although invented decades ago, is an amazing environment for efficiently performing tedious but essential data science tasks. By combining small, powerful, command-line tools (like parallel, jq, and csvkit), you can quickly scrub and explore your data and hack together prototypes.
This hands-on workshop is based on the O’Reilly book Data Science at the Command Line, written by our CEO Jeroen Janssens. You’ll learn how to build fast data pipelines, how to leverage R and Python at the command line, and how to quickly visualise data. No prior knowledge about the unix command line is required.
By the end of this workshop you will have a solid understanding of how to integrate the command line in your data science workflow. Even if you’re already comfortable processing data with, for example, R or Python, being able to also leverage the power of the command line can make you a more effective and efficient data scientist.
What you’ll learn
- Automate tedious tasks
- Parallelise and distribute your tasks to multiple cores and machines
- Convert your existing code to reusable command-line tools
- Easily inspect, transform, and visualise data
- Apply a variety of supervised and unsupervised machine learning algorithms
Schedule
Day 1:
- Introduction
- What is the command line?
- Why learn the command line for doing data science?
- A real-world data science use case
- Getting up and running with the Docker image
- Essential concepts of the unix command line
- Running command-line tools
- Combining command-line tools
- Redirecting input and output
- Working with files
- Getting help
- Obtaining data from logs, spreadsheets, and databases
- Downloading data from the Internet and accessing APIs using
curl - Transforming data with filters such as
cut,paste,grep, andsed - Processing other data formats efficiently
- JSON with
jq - CSV with
csvkit - HTML with
pup - XML with
xmlstarlet
- JSON with
Day 2:
- Running
Rfrom the command line - Visualising data from the command line
- Scatter plot
- Histogram
- Bar chart
- Geographic visualisation
- Parallelising and distributing data-intensive pipelines
- Creating reusable command-line tools
- Automate things in a Bash script
- Convert your existing code to a command-line tool
- Processing arguments
- Working with streaming data
- Applying machine learning
- Outlier detection
- Dimensionality reduction
- Classification
- Regression
- Conclusion
Recommended preparation
Participants are kindly requested to have the following items installed prior to the start of the workshop:
- Docker Desktop for Windows or for Mac or for Ubuntu
- The docker image, by running:
docker pull datascienceworkshops/data-science-at-the-command-line
About your instructor
Jeroen is an RStudio Certified Instructor who enjoys visualizing data, building machine learning models, and automating things using either Python, R, or Bash. Previously, he was an assistant professor at Jheronimus Academy of Data Science and a data scientist at Elsevier in Amsterdam and various startups in New York City. He is the author of Data Science at the Command Line. Jeroen holds a PhD in machine learning from Tilburg University and an MSc in artificial intelligence from Maastricht University.
Clients
We’ve previously delivered this workshop at:
Photos and testimonials
“Great workshop! Very well done and very useful information delivered in an excellent and interactive manner. Jeroen anticipated very well on the different knowledge levels within the group. I would highly recommend the Data Science at the Command Line workshop to anyone that is interested in either kickstarting their command-line experiences or improving their data science with Unix power tools.”
“As a seasoned UNIX command line adept, I didn’t expect to learn much from a Data Science at the Command Line workshop. I was wrong! Over the years, many new tools have become available that I didn’t know about, and that can be combined with traditional tools in new ways.
Since attending the workshop, I have been able to simplify and improve the efficiency of many of the scripts I use on a daily basis. Recommended for anyone working from the command line, newbies and ninjas alike!”
“Besides demonstrating a good knowledge and experience in command-line tools for data science, the instructor had very good training skills, clear communication, and managed to adapt the level of the training to the level of the audience, which is not always easy!”
Sign up
Related workshops
