Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DOv2 catalog discovery (gather feedback)

See original GitHub issue

The objective of this issue is to describe the DO catalog as we have defined in the DO doc


The objective is to allow the user to explore what

  • Datasets
  • Geographies
  • Variables / Variables groups

we have available on CARTO in a similar way as we do on the web, but programmatically.

It is public and the catalog will be accessible without an API key.


From the main class of DO, you can get all the countries or all the categories:

DO.countries => [Country, Country, ...] #Static
DO.categories => [Category, Category, ...] #Static

From every Country or Category, you can access to datasets, geographies and categories/countries:

Country.categories => [Category, Category, ...] 
Country.datasets => [Dataset, Dataset, ...]
Country.geographies => [Geography, Geography, ...]
Category.datasets => [Dataset, Dataset, ...]
Category.countries => [Country, Country, ...]
Category.geographies => [Geography, Geography, ...]

A from a Dataset you can access to the variables:

Dataset.variables = [Variable, Variable, ...]

This is the main idea. Get more info checking the doc.


Inside the data namespace, we are going to create a new one called catalog

Classes structure

First of all, in the doc, check the backend classes in the DO metadata to start creating the big picture.

In CARTOframes, (of course, it could be changed in the implementation) we expect to have the following structure:

  • data
    • Catalog
      • Dataset
      • Country
      • Variable
      • Category
      • Geography

And probably, for each class, we will need a list class

List classes

We want to avoid methods usage, making the discovery cleaner, and at the same time, we want to support the following case:

Dataset.variables => [Variable, Variable, ...]
Dataset.variables.starred => [Variable, Variable, ...]

The idea here is to create a new list class called, for example, Variables extending the Python list class to allow returning the list (Dataset.variables) and also adding methods (Dataset.variables.starred)

Variable and Variables group

A Dataset has variables. The variables are the types/groups of information of the dataset, for example: total population, temperature, …

In the same way, a dataset can have a group of variables, for example: by age, by gender, … A varibles group has several variables, for the case of by age for example: 0-20, 20-40, 40-60, 60+

Also, we are going to have starred variables.

How it works in the catalog

Dataset.variables => returns the first level of variables in the dataset
Variable.children => returns array of variables (if it is a group) or None (if it is a variable)

Getting a element from a list

For example, to get a Country from a countries list:

DO.countries => [Country, Country, ...] #Static

we can do it from several ways (from one or from every one):

  • using the list index:
country = DO.countries[0]
  • static method:
country = Country.get(<country_id>) #Static
  • magic method from the list:
country = DO.countries.get(<country_id>)

PS: Is Dataset from DO the same as the current Dataset from CARTOframes? Good question!!

cc @alasarr @alrocar @andy-esch

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:8 (8 by maintainers)

github_iconTop GitHub Comments

alasarrcommented, Aug 14, 2019

As a quick view, I will remove the download part of this task, since it will involve all the workflow with service accounts and it’s not a discovery method it’s more an access method.

alrocarcommented, Sep 11, 2019

We are moving this to in review let’s close it or freeze it if we don’t gather more feedback during this week.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Citizen Review of Police : Approaches and Implementation
collect data analyze data interpret and report findings. Select Program Type. Review existing program models and materials chapter 3.
Read more >
This online catalog represents the most accurate information on Washington & Jefferson College available at the time of posting. The College reserves the...
Read more >
Australopithecus afarensis - Smithsonian's Human Origins
History of Discovery: The species was formally named in 1978 following a wave of fossil discoveries at Hadar, Ethiopia, and Laetoli, Tanzania.
Read more >
How do data catalogs simplify search & discovery? - Alation
A data catalog makes it easy to search all enterprise data to discover exactly what you need. Learn about the key features of...
Read more >
Discovery Cove FAQ - Plan Your Discovery Cove Trip
Planning a trip to Discovery Cove Orlando? Our FAQs are here to answer your top questions and help you plan a relaxing, one...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found