migrated pipeline execution fails with "usage: bootstrapper.py [-h] -e COS-ENDPOINT -b COS-BUCKET -d COS-DIRECTORY -t..."

1.0.0 Beta 1

I’ve migrated a pipeline that was created using 1.0.0 Beta 0 (and ran successfully in that version). Pipeline execution fails in 1.0.0 Beta 1 and the Kubeflow log displays the following:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2
                                 Dload  Upload   Total   Spent    Left  Speed
3

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  8928  100  8928    0     0  45551      0 --:--:-- --:--:-- --:--:-- 45551
4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
5
                                 Dload  Upload   Total   Spent    Left  Speed
6

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  1289  100  1289    0     0   5912      0 --:--:-- --:--:-- --:--:--  5912
7
Collecting packaging
8
  Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
9
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from packaging) (1.14.0)
10
Collecting pyparsing>=2.0.2
11
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
12
Installing collected packages: pyparsing, packaging
13
Successfully installed packaging-20.4 pyparsing-2.4.7
14
WARNING: You are using pip version 20.0.2; however, version 20.1.1 is available.
15
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
16
Collecting ipykernel==5.3.0
17
  Downloading ipykernel-5.3.0-py3-none-any.whl (119 kB)
18
Collecting ipython==7.15.0
19
  Downloading ipython-7.15.0-py3-none-any.whl (783 kB)
20
Collecting ipython-genutils==0.2.0
21
  Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
22
Collecting jupyter-client==6.1.3
23
  Downloading jupyter_client-6.1.3-py3-none-any.whl (106 kB)
24
Collecting jupyter-core==4.6.3
25
  Downloading jupyter_core-4.6.3-py2.py3-none-any.whl (83 kB)
26
Collecting minio==5.0.10
27
  Downloading minio-5.0.10-py2.py3-none-any.whl (75 kB)
28
Collecting nbclient==0.3.1
29
  Downloading nbclient-0.3.1-py3-none-any.whl (70 kB)
30
Collecting nbconvert==5.6.1
31
  Downloading nbconvert-5.6.1-py2.py3-none-any.whl (455 kB)
32
Collecting nbformat==5.0.7
33
  Downloading nbformat-5.0.7-py3-none-any.whl (170 kB)
34
Collecting papermill==2.1.1
35
  Downloading papermill-2.1.1-py3-none-any.whl (32 kB)
36
Collecting prompt-toolkit==3.0.5
37
  Downloading prompt_toolkit-3.0.5-py3-none-any.whl (351 kB)
38
Collecting pyzmq==19.0.1
39
  Downloading pyzmq-19.0.1-cp37-cp37m-manylinux1_x86_64.whl (1.1 MB)
40
Collecting requests==2.23.0
41
  Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
42
Collecting tornado==6.0.4
43
  Downloading tornado-6.0.4.tar.gz (496 kB)
44
Collecting traitlets==4.3.3
45
  Downloading traitlets-4.3.3-py2.py3-none-any.whl (75 kB)
46
Collecting urllib3==1.25.9
47
  Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
48
Collecting jedi>=0.10
49
  Downloading jedi-0.17.1-py2.py3-none-any.whl (1.4 MB)
50
Collecting backcall
51
  Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
52
Collecting decorator
53
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
54
Collecting pickleshare
55
  Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
56
Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/site-packages (from ipython==7.15.0) (46.0.0)
57
Collecting pexpect; sys_platform != "win32"
58
  Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
59
Collecting pygments
60
  Downloading Pygments-2.6.1-py3-none-any.whl (914 kB)
61
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/site-packages (from jupyter-client==6.1.3) (2.8.1)
62
Collecting configparser
63
  Downloading configparser-5.0.0-py3-none-any.whl (22 kB)
64
Requirement already satisfied: pytz in /usr/local/lib/python3.7/site-packages (from minio==5.0.10) (2019.3)
65
Collecting certifi
66
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
67
Collecting nest-asyncio
68
  Downloading nest_asyncio-1.3.3-py3-none-any.whl (4.7 kB)
69
Collecting async-generator
70
  Downloading async_generator-1.10-py3-none-any.whl (18 kB)
71
Collecting defusedxml
72
  Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
73
Collecting entrypoints>=0.2.2
74
  Downloading entrypoints-0.3-py2.py3-none-any.whl (11 kB)
75
Collecting mistune<2,>=0.8.1
76
  Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
77
Collecting pandocfilters>=1.4.1
78
  Downloading pandocfilters-1.4.2.tar.gz (14 kB)
79
Collecting jinja2>=2.4
80
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
81
Collecting bleach
82
  Downloading bleach-3.1.5-py2.py3-none-any.whl (151 kB)
83
Collecting testpath
84
  Downloading testpath-0.4.4-py2.py3-none-any.whl (163 kB)
85
Collecting jsonschema!=2.5.0,>=2.4
86
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
87
Collecting pyyaml
88
  Downloading PyYAML-5.3.1.tar.gz (269 kB)
89
Collecting ansiwrap
90
  Downloading ansiwrap-0.8.4-py2.py3-none-any.whl (8.5 kB)
91
Collecting click
92
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
93
Collecting tenacity
94
  Downloading tenacity-6.2.0-py2.py3-none-any.whl (24 kB)
95
Collecting black
96
  Downloading black-19.10b0-py36-none-any.whl (97 kB)
97
Collecting tqdm>=4.32.2
98
  Downloading tqdm-4.46.1-py2.py3-none-any.whl (63 kB)
99
Collecting wcwidth
100
  Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
101
Collecting chardet<4,>=3.0.2
102
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
103
Collecting idna<3,>=2.5
104
  Downloading idna-2.9-py2.py3-none-any.whl (58 kB)
105
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from traitlets==4.3.3) (1.14.0)
106
Collecting parso<0.8.0,>=0.7.0
107
  Downloading parso-0.7.0-py2.py3-none-any.whl (100 kB)
108
Collecting ptyprocess>=0.5
109
  Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
110
Collecting MarkupSafe>=0.23
111
  Downloading MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl (27 kB)
112
Collecting webencodings
113
  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
114
Requirement already satisfied: packaging in /usr/local/lib/python3.7/site-packages (from bleach->nbconvert==5.6.1) (20.4)
115
Collecting pyrsistent>=0.14.0
116
  Downloading pyrsistent-0.16.0.tar.gz (108 kB)
117
Collecting attrs>=17.4.0
118
  Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
119
Collecting importlib-metadata; python_version < "3.8"
120
  Downloading importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
121
Collecting textwrap3>=0.9.2
122
  Downloading textwrap3-0.9.2-py2.py3-none-any.whl (12 kB)
123
Collecting regex
124
  Downloading regex-2020.6.8-cp37-cp37m-manylinux2010_x86_64.whl (661 kB)
125
Collecting appdirs
126
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
127
Collecting toml>=0.9.4
128
  Downloading toml-0.10.1-py2.py3-none-any.whl (19 kB)
129
Collecting pathspec<1,>=0.6
130
  Downloading pathspec-0.8.0-py2.py3-none-any.whl (28 kB)
131
Collecting typed-ast>=1.4.0
132
  Downloading typed_ast-1.4.1-cp37-cp37m-manylinux1_x86_64.whl (737 kB)
133
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/site-packages (from packaging->bleach->nbconvert==5.6.1) (2.4.7)
134
Collecting zipp>=0.5
135
  Downloading zipp-3.1.0-py3-none-any.whl (4.9 kB)
136
Building wheels for collected packages: tornado, pandocfilters, pyyaml, pyrsistent
137
  Building wheel for tornado (setup.py): started
138
  Building wheel for tornado (setup.py): finished with status 'done'
139
  Created wheel for tornado: filename=tornado-6.0.4-cp37-cp37m-linux_x86_64.whl size=428918 sha256=9805b80c32bf46ee274bbc7f656940e91278ea5a31eddd505d95394ecace71d2
140
  Stored in directory: /root/.cache/pip/wheels/7d/14/fa/d88fb5da77d813ea0ffca38a2ab2a052874e9e1142bad0b348
141
  Building wheel for pandocfilters (setup.py): started
142
  Building wheel for pandocfilters (setup.py): finished with status 'done'
143
  Created wheel for pandocfilters: filename=pandocfilters-1.4.2-py3-none-any.whl size=7856 sha256=2a663459f6b6555f2e354153fae944c5b5ffa3e7595f078dc0479d5020b5d49e
144
  Stored in directory: /root/.cache/pip/wheels/63/99/01/9fe785b86d1e091a6b2a61e06ddb3d8eb1bc9acae5933d4740
145
  Building wheel for pyyaml (setup.py): started
146
  Building wheel for pyyaml (setup.py): finished with status 'done'
147
  Created wheel for pyyaml: filename=PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl size=471261 sha256=05537babf15d4352ac5227c4afa79da0fdb6b2d4883fe3a8448ff5889544cb98
148
  Stored in directory: /root/.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653
149
  Building wheel for pyrsistent (setup.py): started
150
  Building wheel for pyrsistent (setup.py): finished with status 'done'
151
  Created wheel for pyrsistent: filename=pyrsistent-0.16.0-cp37-cp37m-linux_x86_64.whl size=124352 sha256=80764fffb8cbbf64aba12846693b6d4c422e8076523af73a5b44ac12202c42d5
152
  Stored in directory: /root/.cache/pip/wheels/22/52/11/f0920f95c23ed7d2d0b05f2b7b2f4509e87a20cfe8ea43d987
153
Successfully built tornado pandocfilters pyyaml pyrsistent
154
Installing collected packages: tornado, ipython-genutils, decorator, traitlets, jupyter-core, pyzmq, jupyter-client, parso, jedi, backcall, pickleshare, wcwidth, prompt-toolkit, ptyprocess, pexpect, pygments, ipython, ipykernel, configparser, urllib3, certifi, minio, pyrsistent, attrs, zipp, importlib-metadata, jsonschema, nbformat, nest-asyncio, async-generator, nbclient, defusedxml, entrypoints, mistune, pandocfilters, MarkupSafe, jinja2, webencodings, bleach, testpath, nbconvert, pyyaml, textwrap3, ansiwrap, click, chardet, idna, requests, tenacity, regex, appdirs, toml, pathspec, typed-ast, black, tqdm, papermill
155
Successfully installed MarkupSafe-1.1.1 ansiwrap-0.8.4 appdirs-1.4.4 async-generator-1.10 attrs-19.3.0 backcall-0.2.0 black-19.10b0 bleach-3.1.5 certifi-2020.6.20 chardet-3.0.4 click-7.1.2 configparser-5.0.0 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 idna-2.9 importlib-metadata-1.7.0 ipykernel-5.3.0 ipython-7.15.0 ipython-genutils-0.2.0 jedi-0.17.1 jinja2-2.11.2 jsonschema-3.2.0 jupyter-client-6.1.3 jupyter-core-4.6.3 minio-5.0.10 mistune-0.8.4 nbclient-0.3.1 nbconvert-5.6.1 nbformat-5.0.7 nest-asyncio-1.3.3 pandocfilters-1.4.2 papermill-2.1.1 parso-0.7.0 pathspec-0.8.0 pexpect-4.8.0 pickleshare-0.7.5 prompt-toolkit-3.0.5 ptyprocess-0.6.0 pygments-2.6.1 pyrsistent-0.16.0 pyyaml-5.3.1 pyzmq-19.0.1 regex-2020.6.8 requests-2.23.0 tenacity-6.2.0 testpath-0.4.4 textwrap3-0.9.2 toml-0.10.1 tornado-6.0.4 tqdm-4.46.1 traitlets-4.3.3 typed-ast-1.4.1 urllib3-1.25.9 wcwidth-0.2.5 webencodings-0.5.1 zipp-3.1.0
156
WARNING: You are using pip version 20.0.2; however, version 20.1.1 is available.
157
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
158
ansiwrap==0.8.4
159
appdirs==1.4.4
160
async-generator==1.10
161
attrs==19.3.0
162
backcall==0.2.0
163
black==19.10b0
164
bleach==3.1.5
165
certifi==2020.6.20
166
chardet==3.0.4
167
click==7.1.2
168
configparser==5.0.0
169
decorator==4.4.2
170
defusedxml==0.6.0
171
entrypoints==0.3
172
idna==2.9
173
importlib-metadata==1.7.0
174
ipykernel==5.3.0
175
ipython==7.15.0
176
ipython-genutils==0.2.0
177
jedi==0.17.1
178
Jinja2==2.11.2
179
jsonschema==3.2.0
180
jupyter-client==6.1.3
181
jupyter-core==4.6.3
182
MarkupSafe==1.1.1
183
minio==5.0.10
184
mistune==0.8.4
185
nbclient==0.3.1
186
nbconvert==5.6.1
187
nbformat==5.0.7
188
nest-asyncio==1.3.3
189
numpy==1.18.2
190
packaging==20.4
191
pandas==1.0.3
192
pandocfilters==1.4.2
193
papermill==2.1.1
194
parso==0.7.0
195
pathspec==0.8.0
196
pexpect==4.8.0
197
pickleshare==0.7.5
198
prompt-toolkit==3.0.5
199
ptyprocess==0.6.0
200
Pygments==2.6.1
201
pyparsing==2.4.7
202
pyrsistent==0.16.0
203
python-dateutil==2.8.1
204
pytz==2019.3
205
PyYAML==5.3.1
206
pyzmq==19.0.1
207
regex==2020.6.8
208
requests==2.23.0
209
six==1.14.0
210
tenacity==6.2.0
211
testpath==0.4.4
212
textwrap3==0.9.2
213
toml==0.10.1
214
tornado==6.0.4
215
tqdm==4.46.1
216
traitlets==4.3.3
217
typed-ast==1.4.1
218
urllib3==1.25.9
219
wcwidth==0.2.5
220
webencodings==0.5.1
221
zipp==3.1.0
222
usage: bootstrapper.py [-h] -e COS-ENDPOINT -b COS-BUCKET -d COS-DIRECTORY -t
223
                       COS-DEPENDENCIES-ARCHIVE -i NOTEBOOK [-p OUTPUTS]
224
                       [-l INPUTS]
225
bootstrapper.py: error: unrecognized arguments: data/noaa-weather-data-jfk-airport/jfk_weather_cleaned.csv
226
Package not found. Installing ipykernel package with version 5.3.0...
227
Package not found. Installing ipython package with version 7.15.0...
228
Package not found. Installing ipython-genutils package with version 0.2.0...
229
Package not found. Installing jupyter-client package with version 6.1.3...
230
Package not found. Installing jupyter-core package with version 4.6.3...
231
Package not found. Installing minio package with version 5.0.10...
232
Package not found. Installing nbclient package with version 0.3.1...
233
Package not found. Installing nbconvert package with version 5.6.1...
234
Package not found. Installing nbformat package with version 5.0.7...
235
Package not found. Installing papermill package with version 2.1.1...
236
Package not found. Installing prompt-toolkit package with version 3.0.5...
237
Package not found. Installing pyzmq package with version 19.0.1...
238
Package not found. Installing requests package with version 2.23.0...
239
Package not found. Installing tornado package with version 6.0.4...
240
Package not found. Installing traitlets package with version 4.3.3...
241
Package not found. Installing urllib3 package with version 1.25.9...
242
Imports Complete.....
243
Parsing Arguments.....

[lresende]

@ptitzler, this is a regression on kfp-notebook, which I have an open pr for that. I will leave it open for the rest of the day in case anyone has some feedback, otherwise, I will create a new kfp-notebook release tonight.

[lresende]

@kevin-bates , sorry, I should have closed this. It has been fixed in the latest kfp-notebook release `0.10.1’