question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Include in User Guide / sample scripts how to save to Cosmos DB

See original GitHub issue

While the example to write to Cosmos DB is in CosmosDBDataFrameSpec.scala - this is not at all obvious. Should provide write example directly in readme.md and user guide.

To save a DataFrame to Cosmos DB based on the flights example used thorughout this documentation:

// Import SaveMode so you can Overwrite, Append, ErrorIfExists, Ignore
import org.apache.spark.sql.{Row, SaveMode, SparkSession}

// Create new DataFrame `df` which has slightly flights information 
// i.e. change the delay value to -999
val df = spark.sql("select -999 as delay, distance, origin, date, destination from c limit 5")

// Save to Cosmos DB (using Append in this case)
//    Ensure the baseConfig contains a Read-Write Key
//    The key provided in our examples is a Read-Only Key
df.write.mode(SaveMode.Append).cosmosDB(baseConfig)

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:16 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
dennygleecommented, Apr 6, 2018

Forgot to note here’s the Python version of it:

df.write.format('com.microsoft.azure.cosmosdb.spark').mode('overwrite').options(**writeConfig).save()

thanks @jangcy !

1reaction
dennygleecommented, Apr 5, 2018

@ttelfer for **writeConfig, please refer to https://github.com/Azure/azure-cosmosdb-spark/wiki/Configuration-references. We’re in the process of updating our documentation to help find these things easier.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Quickstart - Azure Cosmos DB for NoSQL client library for .NET
On the Select API option page, select the Create option within the NoSQL section. Azure Cosmos DB has six APIs: NoSQL, MongoDB, PostgreSQL, ......
Read more >
An Introduction and Tutorial for Azure Cosmos DB - InfoQ
This tool will allow you to develop the queries that you would want to subsequently use from your scripts or reporting applications to...
Read more >
azure-cosmosdb-spark - Scaladex
Working with our samples. Included in this GitHub repository are a number of sample notebooks and scripts that you can utilize: On-Time Flight...
Read more >
Azure Cosmos DB Tutorial | Globally distributed NoSQL ...
Azure Cosmos DB is globally distributed and highly responsive database in the cloud. With just few clicks you can get multi-regional ...
Read more >
Load Data Into Cosmos DB with ADF
In the Azure Cosmos DB blade, locate and select the Overview link on the left side of the blade. At the top select...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found