docs: Create and Populate How-To Guides

Create+Populate How-To Guides

There is a lack of how to guides on the website. Let’s make a list of them and start populating a useful section for code schematics.

I hope to keep this issue up as a rolling list of How to Guides, so if you stumble on this and your need isn’t included in the docs or this list, then comment your request here!

v3.2

• How to compute the top k records
• How to register tables in a DuckDB backend #4386
• How to ffill and bfill using window functions (source) #4402

v4.0

• How to join a table expression to a pandas dataframe (merged) (docs PR)
• (WIP) How to chain ibis expressions (_ API) (#4914)
• How to sessionize a log of events based on the elapsed time between successive transactions for a give user identifier (#4843).
• How to convert column structures into columns (unpack)

v4.1+

• How to create a table with a user-defined schema and efficiently load a large CSV file into it (handling common annoying things like the choice of the separator and custom date / time formats).
• How to efficiently sample random rows from a large table (or expr) to quickly compute approximate descriptive statistics such as quantiles.
• How to perform an asof_join (source)
• How to interpolate dates (and/or other sequences)
• [maybe?] How to use Ibis to generate and save Substrait/SQL
• How to create a backend (incl. a section on SQLAlchemy backends specifically, but should focus on min requirements to get one up and running w/o SQLAlch)
• How to chain joins (unknown if this should even be a how-to guide, but if #4309 is a fact of life then we should add this)

Let me know if you have any ideas and I’ll add them to the list.

[ogrisel]

Here are some more ideas:

• How to create a table with a user-defined schema and efficiently load a large CSV file into it (handling common annoying things like the choice of the separator (and maybe quotes) and custom date / time formats).

• How to sessionize a log of events based on the elapsed time between successive transactions for a give user identifier.

• How to efficiently sample random rows from a large table (or expr) to quickly compute approximate descriptive statistics such as quantiles.

For the first item about CSV loading, I don’t know if there is a standard way to use the COPY statements of popular databases from ibis. Personally I used a raw_sql command with duckdb to be able to specify a custom date format in a recent experiment.

The second is about event-based data and should probably cross-reference #4402 (about ffil).

[jcmkk3]

Below is a list of some resources that I think are good examples to better learn tools in the data ecosystem. Maybe some parts of these could be translated to use Ibis. Sometimes one of the hardest parts is to just figure out a good narrative flow or interesting example use case when writing tutorials.

• Tidy Tuesday Project Contains lots of interesting datasets. Both raw and “clean” data, along with the cleaning scripts that produced the clean versions.
• Tidy Tuesday Screencasts - David Robinson (video) Good examples of EDA workflows.
• Tidy Tuesday Screencasts - David Robinson (code)
• Intro to VisiData Good example of an interesting, but well paced introduction to the tool.
• Calm Code - Pandas Pipe
• Computational Notebooks - palewire Lots of good examples of real world data wrangling and analysis.
• First Python Notebook Another interesting, but well paced introduction to data tooling.
• R for Data Science A classic for a reason. Lots of good examples as well as ways to think about data analysis tasks.
• Reproducible Data Analysis in Jupyter A good example of a workflow for exploring data in python.
• Arquero notebook collection Some nice examples of interleaving a bit of a story and interesting tidbits while teaching a tool.

Many of these examples have other elements outside of just data manipulation like visualization or maybe some stats/machine learning. It could get a little bit tricky to choose which other libraries to feature, but I do think that it is useful to see how different tools interact.