Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KeyError: 'type' when publishing CSV to Enterprise Portal

See original GitHub issue

I have a CSV file that I cannot publish to ArcGIS Enterprise Portal from a ArcGIS Pro Notebook. I have a stand-alone script that requests JSON from a URL and writes it to a CSV. The content has no special characters that I’m aware of. When opening the CSV from my stand-alone script, I encoded it to utf-8. But, when subsequently attempting to publish to the Portal I get the KeyError. I’m suspecting it’s a character encoding issue in the CSV, but I’m in no way certain.

To Reproduce I’m not sure if this is necessary but here is my script that writes to the CSV. It’s completely reproducible assuming you have the relevant modules.

import json,requests
import os,sys
import csv

#change file path to new directory
path_to_file = os.path.join(r'C:\Users\jpilbeam\Downloads', 'c19_Vaccine_Current.csv')
#idph url goes here
idph_data = ''

##get json data from idph
response = requests.get(idph_data,verify=True)
#read the json response and keep the VaccineAdministration part
data = response.json()['VaccineAdministration']

#write to file
with open(path_to_file, 'w', newline='', encoding='UTF-8') as csvfile:
    f = csv.writer(csvfile) 
    #write the headers of the csv file
    f.writerow(['County','AdminCount','AdminCountChange', 'RollAvg', 'AllocDoses', 'FullyVaccinated', 
                'FullyVaccinatedChange', 'ReportDate', 'Pop', 'PctVaccinated', 'LHDInventory', 'CommInventory',
                'TotalInventory', 'InventoryDate'])
    for elem in data:
        #get the values for all the keys (i.e. CountyName, AdministeredCount, etc...)
        f.writerow([elem['CountyName'], elem['AdministeredCount'], elem['AdministeredCountChange'], 
                    elem['AdministeredCountRollAvg'], elem['AllocatedDoses'], 
                    elem['PersonsFullyVaccinated'], elem['PersonsFullyVaccinatedChange'], 
                    elem['Report_Date'], elem['Population'], elem['PctVaccinatedPopulation'], 
                    elem['LHDReportedInventory'], elem['CommunityReportedInventory'], 
                    elem['TotalReportedInventory'], elem['InventoryReportDate']])>

In a Pro Notebook I then run the following code on the resulting CSV. It’s when I try publishing that I get the KeyError.

from IPython.display import display
from arcgis.gis import GIS
import os
gis = GIS('Home') #uses current Notebook as workspace

csv_file = r'C:\Users\jpilbeam\Downloads\c19_Vaccine_Current.csv' #path to CSV
csv_item = gis.content.add({}, csv_file) #add CSV to Enterprise Portal
display(csv_item) #display it here 

#location parameter needs to be set to "none" when publishing to Enterprise Portal
csv_item.publish(publish_parameters=params) #publish to Enterprise Portal

KeyError                                  Traceback (most recent call last)
In  [7]:
Line 3:     csv_item.publish(publish_parameters=params) #publish to Enterprise Portal

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\, in publish:
Line 10353: elif not buildInitialCache and ret[0]['type'].lower() == 'image service':

KeyError: 'type'

If it makes any difference I was able to publish only by opening the CSV in Notepad --> setting Encoding to utf-8 --> giving the filename a *.csv extension --> then saving. Needless to say, I don’t intend to go this route. image

Expected behavior I expected the CSV to publish once I set the encoding to utf-8 in the with open() as csvfile part. I’ve attached my posts from GeoNet and StackExchange relating to this issue. It shows how I’ve come to this point.


  • OS: Windows 10
  • Browser FireFox
  • Python API Version 1.8.4

Additional context My related posts on the issue:

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:20 (4 by maintainers)

github_iconTop GitHub Comments

UnderJollyRogercommented, Jul 14, 2021

@achapkowski @Detteor After some additional help from an ESRI analyst I have a more or less stable script to publish a CSV to Portal as a feature service. And because it’s a reoccurring process there’s an additional script that overwrites the feature service. See this post.

Detteorcommented, Apr 6, 2021


@Detteor By the way, this is from an ESRI blog: it is not recommended that you have both Anaconda and ArcGIS Pro installed on a single machine

This Definitely might have been the case when using ArcGIS Pro 2.2, but now with the metapackages and being able to install arcpy in an environment it doesn’t seem as impactful anymore. ENV and PATH variables are always a pain in windows, so I don’t even let conda set its own variables. Also, they don’t want you to mess with the base install since that can cause a lot of issues, this is one of the reasons that they don’t let you change the base install for ArcGIS Pro.

@Detteor Good to know. I’m already using a conda environment in Pro 2.7.2 as fas as I can tell. I didn’t bother going through the Anaconda site, as I installed the API with Pro. I’m connected to the same environment in my IDE. This is from Visual Studio 2019.

When I start to work on new projects, I always use a new environment that has the latest package versions. I actually never touch the conda environment that comes with ArcGIS Pro since I have found that it causes a lot of problems. I do now that the version of ArcGIS API that comes with ArcGIS Pro is locked at 1.8.3 due to compatibility issues. You might want to create a whole new environment that isn’t going to be used in ArcGIS Pro and that will only be used in your IDE.

You can create an env.yml file that you can use to install the packages that you want in your new environment. You will need to activate your environment and type conda env export > env.yml, this will save a yml file in your current working directory of the packages currently installed. You can also use conda env export --from-history > env.yml, this will export only the packages that you have installed.Then you can use conda create -n myenv python=3.7 use 3.7 since that is what arcpy is running in ArcGIS Pro.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Publish CSV to Enterprise Portal Errors - Esri Community
I'm used to my Visual Studio 2019 IDE so I brought the code over from the Pro Notebook it was in. And the...
Read more >
Publish CSV to Enterprise Portal as table - GIS Stack Exchange
1 Answer 1 · Open the CSV in a text editor (I used Notepad). · Save it. Use a .csv file extension type...
Read more >
Why do I receive a KeyError when I write my data to csv? What ...
1 Answer 1 ... You are getting the error because one or more records in the r.get("alternatives") does not have confidence key -...
Read more >
Troubleshoot—ArcGIS Online Help | Documentation
Publishing a service definition (.sd) file on ArcGIS Online fails with the message Unable to publish item. This Item can't be published to...
Read more >
Changelog — Python 3.11.1 documentation
gh-87604: Avoid publishing list of active per-interpreter audit hooks via the ... gh-98852: Fix subscription of type aliases containing bare ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found