Super slow

This Script takes almost 20 minutes to parse through 186 rows.

bufsize = 1
with open('/cygdrive/c/hsmtest_adep/final_batch.sh', 'w', bufsize) as f:
    for row in range(3, 186):
        sentry = ''
        counter = 0
        while not sentry and counter < 3:
            try:
                gc = gspread.login('xxxxx', 'xxxxx') # If i do this outside the loop i see intermittent failures
                sht = gc.open_by_key('xxxx')
                worksheet = sht.worksheet("xxxxxx")
                values_list = worksheet.row_values(row)
                sentry = True;
            except:
                print("Row:", row, "Got Worksheet Error, Trying Again...")
                counter += 1

        try:
           print("./test_alias.sh -a", values_list[2])
           if values_list[2]:
               values = "./test_alias.sh -a " + values_list[2] + "\n"
           string = str(values)
           f.write(string)
        except IndexError:
           print("No Cert Alias for row id:", row)

f.close()

[burnash]

Google Spreadsheets API can be pretty slow depending on many factors including your connection speed to Google servers, usage of proxy, etc. In your code snippet I can see gspread.login inside a loop. Please consider pulling this out of the loop because this method is slow. Having failures while connecting to Google API is another matter for creating an issue on GitHub.

Note: please pay attention to the formatting of the Python code you’ve posted. It is unreadable without proper usage of whitespace. Check GitHub manual on how to use Markdown in comments and issues.

[burnash]

@samuelcolvin Interestingly, I’ve just checked the API reference, and I couldn’t find any trace of range parameter in the docs anymore. However the test suite runs just fine.

I’ve debugged the range method to check the data in the server’s response, and for a requested range A1:B5 there was exactly 10 entries in the XML feed:

<ns0:feed xmlns:ns0="http://www.w3.org/2005/Atom" xmlns:ns1="http://a9.com/-/spec/opensearchrss/1.0/" xmlns:ns2="http://schemas.google.com/spreadsheets/2006">
  <ns0:id>https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full</ns0:id>
  <ns0:updated>2015-01-07T00:10:54.135Z</ns0:updated>
  <ns0:category scheme="http://schemas.google.com/spreadsheets/2006" term="http://schemas.google.com/spreadsheets/2006#cell" />
  <ns0:title type="text">Sheet1</ns0:title>
  <ns0:link href="https://docs.google.com/spreadsheets/d/my-key/edit" rel="alternate" type="application/atom+xml" />
  <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full" rel="http://schemas.google.com/g/2005#feed" type="application/atom+xml" />
  <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full" rel="http://schemas.google.com/g/2005#post" type="application/atom+xml" />
  <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/batch" rel="http://schemas.google.com/g/2005#batch" type="application/atom+xml" />
  <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full?range=A1%3AB5&amp;return-empty=true" rel="self" type="application/atom+xml" />
  <ns0:author>
    <ns0:name>my-name</ns0:name>
    <ns0:email>[e-mail]</ns0:email>
  </ns0:author>
  <ns1:totalResults>10</ns1:totalResults>
  <ns1:startIndex>1</ns1:startIndex>
  <ns2:rowCount>100</ns2:rowCount>
  <ns2:colCount>30</ns2:colCount>
  <ns0:entry>
    <ns0:id>https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C1</ns0:id>
    <ns0:updated>2015-01-07T00:10:54.135Z</ns0:updated>
    <ns0:category scheme="http://schemas.google.com/spreadsheets/2006" term="http://schemas.google.com/spreadsheets/2006#cell" />
    <ns0:title type="text">A1</ns0:title>
    <ns0:content type="text">cell_A1</ns0:content>
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C1" rel="self" type="application/atom+xml" />
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C1/aytj4d" rel="edit" type="application/atom+xml" />
    <ns2:cell col="1" inputValue="cell_A1" row="1">cell_A1</ns2:cell>
  </ns0:entry>
  <ns0:entry>
    <ns0:id>https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C2</ns0:id>
    <ns0:updated>2015-01-07T00:10:54.135Z</ns0:updated>
    <ns0:category scheme="http://schemas.google.com/spreadsheets/2006" term="http://schemas.google.com/spreadsheets/2006#cell" />
    <ns0:title type="text">B1</ns0:title>
    <ns0:content type="text">cell_B1</ns0:content>
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C2" rel="self" type="application/atom+xml" />
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R1C2/aotxrg" rel="edit" type="application/atom+xml" />
    <ns2:cell col="2" inputValue="cell_B1" row="1">cell_B1</ns2:cell>
  </ns0:entry>

  <!-- Skip 7 entries -->

  <ns0:entry>
    <ns0:id>https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R5C2</ns0:id>
    <ns0:updated>2015-01-07T00:10:54.135Z</ns0:updated>
    <ns0:category scheme="http://schemas.google.com/spreadsheets/2006" term="http://schemas.google.com/spreadsheets/2006#cell" />
    <ns0:title type="text">B5</ns0:title>
    <ns0:content type="text">cell_B5</ns0:content>
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R5C2" rel="self" type="application/atom+xml" />
    <ns0:link href="https://spreadsheets.google.com/feeds/cells/my-key/od6/private/full/R5C2/aoobhs" rel="edit" type="application/atom+xml" />
    <ns2:cell col="2" inputValue="cell_B5" row="5">cell_B5</ns2:cell>
  </ns0:entry>
</ns0:feed>

I think the range is working correctly, and the returned data is less than in the case of returning the entire sheet. My guess is that in your case it’s not the amount of data responsible for the timing of the range but the fact that there’s a creation of a Cell object for every cell in the range. I’ll try to profile the code to see the real reason. Thank you for noting this!