*** DANGER*** mbed update --clean deletes your local git repos, which should never happen

Hi

I’ve lost 3 days work while sorting through a combination of problems including:

• taking my mbed-os fork prior to the following changes
• rebasing my local git repo including 2 new commits, pushing to developer fork, and undetected failure of github to take the push of the rebase
• removal of mbed-os/tools (un-notified to developer)
• mbed compile --tests problems (failure to find mbed-os/tools/test.py script)
• mbed-cli tool upgrade due to build problems due to removal of mbed-os/tools dir (uncertain whether upgrade was required but was done due to build problems)
• move of mbed-os/storage/cfstore to mbed-os/features/storage/cfstore (un-notified to developer)

The loss was caused by mbed update --clean, which removed some old directories, which resulted in the mbed-os git local repo being removed. The mbed update --clean option should therefore be removed from the tool.

This is a record of what happened (my git bash history)

git commit, rebase, push changes to my remote mbed-os fork

 2143  git commit -m "Cfstore flash test fixes for exmaple1 and misc."
 2144  git status
 2145  history | grep rebase
 2146  git rebase upstream/master
 2147  git status
 2148  git push -f origin master
 2149  popd
 2150  rm -fR .build

The above operation appear to work fine (no error messages or warnings that the rebase has a problem) but the pushd failed to put the changes in the fork. I only later found this out when trying to recover from the mbed update --clean.

 2151  mbed compile -v --tests -m K64F  -t GCC_ARM 2>&1 | tee cfstore_mbedos_blinky5_build_tests_20160616_0957.txt

tried to build the rebased tree but experienced a build problem: mbed-cli tool reports mbed-os\tools dir has been removed. Hence I figure its time to upgrade mbed-cli

 2152  mbed -h
 2153  pushd ..
 2154  ls -al
 2155  pushd mx2_venv/
 2156  ls -al
 2157  pushd work/
 2158  ls -al
 2159  pushd neo/
 2160  ls -al
 2161  python setup.py -h
 2162  python setup.py --help-commands
 2163  popd
 2164  ls -al
 2165  ls -al greentea/
 2166  popd
 2167  pushd Scripts/
 2168  deactivate
 2169  popd
 2170  virtualenv.exe mx2_venv2
 2171  pushd mx2_venv2/Scripts/
 2172  . activate
 2173  cd ..
 2174  ls
 2175  ls -al
 2176  mkdir work
 2177  pushd work/
 2178  git clone [e-mail]:/armmbed/greentea.git
 2179  mbedgt
 2180  mbedgt --version
 2181  pushd greentea/
 2182  ls
 2183  python setup.py install
 2184  mbedgt --version
 2185  popd
 2186  mbed
 2187  git clone [e-mail]:/armmbed/mbed-cli
 2188  pushd mbed-cli/
 2189  ls a-l
 2190  ls -al
 2191  python setup.py install
 2192  mbed --version
 2193  mbed -h
 2194  python setup.py -U install
 2195  python setup.py -h
 2196  ls -al
 2197  less README.md 
 2198  ls -al
 2199  less setup.py 
 2200  q
 2201  type mbed
 2202  popd
 2203  pip install mbed-cli
 2204  pip install mbed-cli --upgrade
 2205  mbed -h
 2206  python
 2207  pushd mbed-cli/
 2208  python setup.py install --record files.txt
 2209  less files.txt 
 2210  cp files.txt files2.txt 
 2211  for f in `cat files.txt `; do echo $f; basename $f; find /c/mbed_tools/Python27/ -iname `basename $f` ; done
 2212  for f in `cat files.txt `; do echo $f; basename $f; done
 2213  for f in `cat files.txt `; do echo $f; $(basename $f); done
 2214  for f in `cat files.txt `; do echo $f; $(basename "$f"); done
 2215  for f in `cat files.txt `; do echo $f; echo $(basename "$f"); done
 2216  for f in `cat files.txt `; do echo $f; filename=$(basename "$fullfile") && echo $filename; done
 2217  for f in `cat files.txt `; do echo $f; filename=$(basename "$f") && echo $filename; done
 2218  a=/tmp/file.txt
 2219  b=$(basename $a)
 2220  echo $f
 2221  echo $b
 2222  a=d:\tmp\file.txt
 2223  b=$(basename $a)
 2224  echo $b
 2225  type mbed
 2226  mbed
 2227  hash mbed
 2228  type mbed
 2229  mbed -h
 2230  popd
 2231  pushd cfstore_mbedos_blinky5
 2232  ls -al
 2233  ls -al .build
 2234  mbed compile -v --tests -m K64F  -t GCC_ARM 2>&1 | tee cfstore_mbedos_blinky5_build_tests_20160616_1028.txt
 2235  type mbed
 2236  mbed -he
 2237  mbed -h
 2238  ls -al
 2239  mbed compile -v --tests -m K64F  -t GCC_ARM 
 2240  mbed -h
 2241  mbed default root .
 2242  mbed compile -v --tests -m K64F  -t GCC_ARM 
 2243  ls -la mbed-os/tools/

The above is the uninstalling of the old version of mbed-cli that had been installed on the path and was being inherited by the virtualenv

Now trying to get a sane tree, I perform the mbed update --clean (big mistake) :

 2244  mbed update --clean
 2245  history
 2246  history | grep origin
 2247  history
 2248  ls a-l
 2249  cat mbed-os.lib 
 2250  mbed
 2251  ls -al mbed-os/features/storage/cfstore/
 2252  mbed compile -v --tests -m K64F  -t GCC_ARM 2>&1 | tee cfstore_mbedos_blinky5_build_tests_20160616_1103.txt
 2253  mbed d:\datastore\public\jobs\yr2016\2253\sdh_dev_mx2\cfstore_mbedos_blinky5 .
 2254  mbed -h
 2255  mbed compile -v --tests -m K64F  -t GCC_ARM

Tried to build again but has a new problem finding the mbed-cli installed in virtualenv on path. Bogdan says mbed new . solves this:

 2256  mbed
 2257  mbed new .
 2258  l s-al
 2259  ls -la
 2260  cat .mbed 
 2261  mbed compile -v --tests -m K64F  -t GCC_ARM
 2262  pushd mbed-os
 2263  ls -al
 2264  cat requirements.txt 
 2265  python setup.py install
 2266  popd
 2267  pip install jinja2
 2268  mbed compile -v --tests -m K64F  -t GCC_ARM
 2269  rm -fR 
 2270  rm -fR .build/
 2271  mbed compile -v --tests -m K64F  -t GCC_ARM 2>&1 | tee cfstore_mbedos_blinky5_build_tests_20160616_1109.txt
 2272  ls -al
 2273  mv main.cpp main_cpp
 2274  mbed compile -v --tests -m K64F  -t GCC_ARM 2>&1 | tee cfstore_mbedos_blinky5_build_tests_20160616_1110.txt
 2275  ls -al

build has worked.

However, I’ve lost my local git repo with my changes, and my github fork doesn’t have my changes (probably because the mbed-os maintainers have moved

mbed-os/storage/cfstore

to

mbed-os/features/storage/cfstore

which has caused my git rebase and push to my mbed-os fork, which was forked before the cfstore dir moved.

[bogdanm]

I agree although git clean -fqd doesn’t require confirmation?

It doesn’t, but that shouldn’t stop us from being more user-friendly if we can (you have to admit that git isn’t exactly user-friendly 😃 )

[bogdanm]

The problem is the mbed cli tool should never delete dirs and especially git local repo trees

This is debatable. git clean will do a very similar thing, destroying your local changes if instructed to do so. This behaviour, while dangerous, is quite useful in some cases. Personally, I think the proper fix for this is an update to README with more emphasis on the dangers of using update --clean. Note that some emphasis already exists. From README.md:

Use these with caution as your uncommitted changes and unpublished libraries cannot be restored.