Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

NASBench201 KeyError

See original GitHub issue

Thanks for sharing the great repo. I tried to access the accuracy of ‘x-valid’ on CIFAR10 dataset but failed because of the key error.

>>> results = api.query_by_index(0, 'cifar10')
>>> results.keys()
dict_keys([777, 888, 999])
>>> results[888].get_eval('x-valid')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yangzhaohui/anaconda3/lib/python3.7/site-packages/nas_201_api/api.py", line 885, in get_eval
    'loss'    : self.eval_losses['{:}@{:}'.format(name,iepoch)],
KeyError: 'x-valid@199'

Issue Analytics

State:
Created 3 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

D-X-Ycommented, Jul 1, 2020

You are welcome! Thanks for using our benchmark 😃

0reactions

zhaohui-yangcommented, Jul 1, 2020

It’s a great work and everyone should use the benchmark to verify their search algorithm!

Top Results From Across the Web

NAS-Bench-201 API and Instruction - GitHub

We propose an algorithm-agnostic NAS benchmark (NAS-Bench-201) with a fixed search space, which provides a unified benchmark for almost any up-to-date NAS ...

nes.darts.baselearner_train.oneshot.darts_wrapper_discrete ...

... i.e. when querying nasbench201 if not nb201: # seed_init can be anything ... seed)] except KeyError: seed_list.remove(seed) seed = seed_list[0] try: ...

KeyError raised when training a neural network with Nolearn

I have solved the problem myself. The data types had to be numpy arrays instead of pandas dataframes.

NAS Benchmarks - Neural Network Intelligence - Read the Docs

The directory that stores the benchmark file can be configured with NASBENCHMARK_DIR environment variable, which defaults to ~/.nni/nasbenchmark . Note that the ...

KeyError error message in master head? - Deep Graph Library

Have a model that works with 0.7 release, then switch to master head b/c we need distributed engative sampler feature.