Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

End-to-End Export Testing [Timebox]

See original GitHub issue

This task is to do end-to-end testing for exports - the goal is to make and document the API calls for the following types of exports. The result of this task is to document the API call made in this issue - e.g. what does the POST call to exports look like?

Export types to test:

  • RGB export
  • false-color composite
  • export all bands
  • NDVI export
  • NDWI export
  • EVI Export

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
pomadchincommented, Jun 14, 2017

Results, Part 1

Some results of exports testing.

Issues realted to export (https://github.com/azavea/raster-foundry/pull/1920, https://github.com/azavea/raster-foundry/issues/1925) and import (https://github.com/azavea/raster-foundry/pull/1899, https://github.com/azavea/raster-foundry/pull/1903) were discovered during the test process, however all in all export routes work as expected and RGB export definitely works.

Export all bands

Is not supported. In the current implementation RAW export would be done if there would be no color correction settings. Right now it’s not possible to delete / reset color correction settings, it means that all exports work in RGB / false-color composite modes.

NDVI, NDWI, EVI export

No supported as well, as AST Export doesn’t support projects. Though in the postman query dump (WARN: headers information should be changed as it’s unique and required for auth) a query with a try to make AST export is done, though never finished succesfull.

RGB export / false-color composite

URLs used:

Current WEB UI:

POST Query:

{
  "organizationId":"dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId":"e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus":"NOTEXPORTED",
  "exportType":"S3",
  "visibility":"PRIVATE",
  "exportOptions":{
    "resolution":16
  }
}

Response:

"JSON decoder exception: DecodingFailure at .stitch: Attempt to decode value on failed cursor"

Default RGB:

POST Query:

{
  "organizationId":"dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId":"e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus":"NOTEXPORTED",
  "exportType":"S3",
  "visibility":"PRIVATE",
  "exportOptions":{
    "resolution":8,
    "stitch": {false | true},
    "crop": {false | true},
    "source": "s3://rasterfoundry-test/daunnc-export-test"
  }
}

Response:

{
  "organizationId":"dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId":"e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus":"NOTEXPORTED",
  "exportType":"S3",
  "visibility":"PRIVATE",
  "exportOptions":{
    "resolution":8,
    "stitch": {false | true},
    "crop": {false | true},
    "source": "s3://rasterfoundry-test/daunnc-export-test"
  }
}

Custom color correction:

Custom Correction POST Query:

{  
   "items":[  
      {  
         "sceneId":"0137e34c-6217-4a90-8724-dca423890fa9",
         "params":{  
            "redBand":3,
            "greenBand":2,
            "blueBand":1,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            },
            "$promise":{  

            },
            "$resolved":true
         }
      },
      {  
         "sceneId":"95d1b9ea-03bb-4fe8-a8be-efbc0da936e0",
         "params":{  
            "redBand":3,
            "greenBand":2,
            "blueBand":1,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            },
            "$promise":{  

            },
            "$resolved":true
         }
      },
      {  
         "sceneId":"88c1809f-535a-4e0b-a59b-41c77d4fd722",
         "params":{  
            "redBand":3,
            "greenBand":2,
            "blueBand":1,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            },
            "$promise":{  

            },
            "$resolved":true
         }
      },
      {  
         "sceneId":"48daa61a-54d0-4b06-8e78-03f66e649d27",
         "params":{  
            "redBand":3,
            "greenBand":2,
            "blueBand":1,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            },
            "$promise":{  

            },
            "$resolved":true
         }
      },
      {  
         "sceneId":"1aed5185-25f0-4c8e-a94d-97bca2c451d0",
         "params":{  
            "redBand":3,
            "greenBand":2,
            "blueBand":1,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            },
            "$promise":{  

            },
            "$resolved":true
         }
      }
   ]
}

Exports POST query:

{
  "organizationId":"dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId":"e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus":"NOTEXPORTED",
  "exportType":"S3",
  "visibility":"PRIVATE",
  "exportOptions":{
    "resolution":8,
    "stitch": false,
    "crop": false,
    "source": "s3://rasterfoundry-test/daunnc-export-test"
  }
}

Response

{
  "id": "fd0ad483-333b-457b-bfee-7ddae8bab28c",
  "createdAt": "2017-06-06T10:34:24.025Z",
  "createdBy": "google-oauth2|108115194407282440652",
  "modifiedAt": "2017-06-06T10:34:24.025Z",
  "modifiedBy": "google-oauth2|108115194407282440652",
  "owner": "google-oauth2|108115194407282440652",
  "organizationId": "dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId": "e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus": "NOTEXPORTED",
  "exportType": "S3",
  "visibility": "PRIVATE",
  "toolRunId": null,
  "exportOptions": {
    "crop": false,
    "source": "s3://rasterfoundry-test/daunnc-export-test",
    "stitch": false,
    "resolution": 8
  }
}

All preset colors (Urban, Water, Natural Color, Vegetation & water, Bare Earth change detection, Atmosphere penetration, Forest Fire, Agriculture) work fine as well, as it’s a sepcial case of the thing above, however testing of them was done as well, and can be reproduced using the postman query dump (WARN: headers information should be changed as it’s unique and required for auth).

The whole process works well: WEBUI => Airflow => EMR (emr step experience some permission issues, though it can be only my personal issue)

0reactions
pomadchincommented, Jun 14, 2017

Results, Part 3 (Part 1 extension)

Addition to show that false color export works.

Urban Color correction:

Color Correction POST Query:

{  
   "items":[  
      {  
         "sceneId":"0137e34c-6217-4a90-8724-dca423890fa9",
         "params":{  
            "redBand":6,
            "greenBand":5,
            "blueBand":4,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            }
         }
      },
      {  
         "sceneId":"95d1b9ea-03bb-4fe8-a8be-efbc0da936e0",
         "params":{  
            "redBand":6,
            "greenBand":5,
            "blueBand":4,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            }
         }
      },
      {  
         "sceneId":"88c1809f-535a-4e0b-a59b-41c77d4fd722",
         "params":{  
            "redBand":6,
            "greenBand":5,
            "blueBand":4,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            }
         }
      },
      {  
         "sceneId":"48daa61a-54d0-4b06-8e78-03f66e649d27",
         "params":{  
            "redBand":6,
            "greenBand":5,
            "blueBand":4,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            }
         }
      },
      {  
         "sceneId":"1aed5185-25f0-4c8e-a94d-97bca2c451d0",
         "params":{  
            "redBand":6,
            "greenBand":5,
            "blueBand":4,
            "gamma":{  
               "enabled":true,
               "redGamma":1.33,
               "greenGamma":1.36,
               "blueGamma":1.34
            },
            "bandClipping":{  
               "enabled":true,
               "redMax":212,
               "greenMax":212,
               "blueMax":212,
               "redMin":77,
               "greenMin":77,
               "blueMin":77
            },
            "sigmoidalContrast":{  
               "enabled":true,
               "alpha":1,
               "beta":0.4
            },
            "tileClipping":{  
               "enabled":true,
               "min":77,
               "max":212
            },
            "equalize":{  
               "enabled":false
            },
            "saturation":{  
               "enabled":false,
               "saturation":1
            },
            "autoBalance":{  
               "enabled":false
            }
         }
      }
   ]
}

Exports POST query:

{
  "organizationId":"dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId":"e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus":"NOTEXPORTED",
  "exportType":"S3",
  "visibility":"PRIVATE",
  "exportOptions":{
    "resolution":8,
    "stitch": false,
    "crop": false,
    "source": "s3://rasterfoundry-test/daunnc-export-test"
  }
}

Response

{
  "id": "fd0ad483-333b-457b-bfee-7ddae8bab28c",
  "createdAt": "2017-06-06T10:34:24.025Z",
  "createdBy": "google-oauth2|108115194407282440652",
  "modifiedAt": "2017-06-06T10:34:24.025Z",
  "modifiedBy": "google-oauth2|108115194407282440652",
  "owner": "google-oauth2|108115194407282440652",
  "organizationId": "dfac6307-b5ef-43f7-beda-b9f208bb7726",
  "projectId": "e565df8c-63c2-4544-b0f9-08ac1900f414",
  "exportStatus": "NOTEXPORTED",
  "exportType": "S3",
  "visibility": "PRIVATE",
  "toolRunId": null,
  "exportOptions": {
    "crop": false,
    "source": "s3://rasterfoundry-test/daunnc-export-test",
    "stitch": false,
    "resolution": 8
  }
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Timeboxing: The Ultimate Strategy to Skyrocket Productivity
When your session ends, mark off one Pomodoro and record what you ... the timeboxing method, here are a few tried and tested...
Read more >
Iteration Planning - TechDocs
The purpose of the iteration planning meeting is for the team to commit to the completion of a set of the highest-ranked product...
Read more >
How do we Manage Bugs in Agile? - Anca Onuta - Medium
Learn how successful Agile Teams handle the bugs with Agile methodologies to be predictable and deliver customer value with their products.
Read more >
Chapter 13: Timeboxing - Agile Business Consortium
Timeboxing is one of DSDM's key practices. DSDM defines a Timebox as a fixed period of time, at the end of which an...
Read more >
What is a Timebox? - Agile Alliance
The critical rule of timeboxed work is that work should stop at the end of the timebox, and review progress: has the goal...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Landsat and Sentinel metadata displays date incorrectly

Next page

Areas of Interest