REST API

A REST API would be useful to access ATM from any other language and device through the web, and also to illustrate how the code is structured and might be extended.

From the project’s readme I get that the internal API’s are going to change and that it consequently might be a bit early to develop a REST API, but I wanted to see what was possible to do with the current code. The API currently serves:

• Various GET endpoints for reading data from the four entities that are currently present in the database
• 1 GET endpoint to run the worker.py script inside the virtualenv as a subprocess and retrieve it’s stdout and stderr
• 1 POST endpoint to send a .csv file with the HTTP request, save the file to the atm/data directory and run enter_data on it.

No modifications were made outside of the rest_api_server.py file except to the requirements.txt file, adding flask and simplejson to the project dependencies.

TODO’s / caveats:

• No AWS integration
• api.py currently does not check if the uploaded filename is already present, so a CSV upload can rewrite a previously sent file with the same name. This needs fixing, but I thought that I should ask first if the atm/data directory is the right one to put new files in, and if storing UUID’s is ok with the project’s design before using them and doing a pull request.

Example api.py usage:

After following the readme’s installation instructions and running python scripts/rest_api_server.py on a separate shell under the virtualenv:

curl localhost:5000/enter_data -F file=@/path/file.csv

It should return:

{"success": true}

To see the created dataset:

curl localhost:5000/datasets/1 | jq

{
  "class_column": "class",
  "description": null,
  "size_kb": 6,
  "test_path": null,
  "k_classes": 3,
  "majority": 0.333333333,
  "d_features": 6,
  "train_path": "FULLPATH/file.csv",
  "id": 1,
  "n_examples": 150,
  "name": "file"
}

To see the created datarun:

curl localhost:5000/dataruns/1 | jq

{
  "status": "pending",
  "start_time": null,
  "description": "uniform__uniform",
  "r_minimum": 2,
  "metric": "f1",
  "budget": 100,
  "selector": "uniform",
  "priority": 1,
  "score_target": "cv_judgment_metric",
  "deadline": null,
  "budget_type": "classifier",
  "id": 1,
  "tuner": "uniform",
  "dataset_id": 1,
  "gridding": 0,
  "k_window": 3,
  "end_time": null
}

To run the worker.py script once:

curl localhost:5000/simple_worker | jq

after a while

{
  "stderr": "",
  "stdout": "huge stdout string with worker.py's output"
}

[RogerTangos]

Hey there - I’m picking up this issue. I think @jobliz’s design was pretty good, and I’m going to propose a few more endpoints and decouple the API from the database

[csala]

The first read-only version of the REST API is already in master, so I’m closing this issue now so future enhancements can be added as new issues.