Error when trying to use BLIP, Dreambooth, Hypernetwork and Embeddings. SD 2.1 - Automatic1111 Updated.

First i have this problem when trying to use BLIP for Captions in Automatic1111

0% 0/770 [00:01<?, ?it/s] Error completing request Arguments: ('/content/gdrive/MyDrive/Dataset', '/content/gdrive/MyDrive/DatasetFinal', 768, 768, 'ignore', False, False, True, False, 0.5, 0.2, True, 0.9, 0.15, 0.5, False) {} Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 45, in f res = list(func(*args, **kwargs)) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 28, in f res = func(*args, **kwargs) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/ui.py", line 19, in preprocess modules.textual_inversion.preprocess.preprocess(*args) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 23, in preprocess preprocess_work(process_src, process_dst, process_width, process_height, preprocess_txt_action, process_flip, process_split, process_caption, process_caption_deepbooru, split_threshold, overlap_ratio, process_focal_crop, process_focal_crop_face_weight, process_focal_crop_entropy_weight, process_focal_crop_edges_weight, process_focal_crop_debug) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 188, in preprocess_work save_pic(focal, index, params, existing_caption=existing_caption) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 83, in save_pic save_pic_with_caption(image, index, params, existing_caption=existing_caption) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 52, in save_pic_with_caption caption += shared.interrogator.generate_caption(image) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/interrogate.py", line 133, in generate_caption caption = self.blip_model.generate(gpu_image, sample=False, num_beams=shared.opts.interrogate_clip_num_beams, min_length=shared.opts.interrogate_clip_min_length, max_length=shared.opts.interrogate_clip_max_length) File "/content/gdrive/MyDrive/sd/stablediffusion/src/blip/models/blip.py", line 156, in generate outputs = self.text_decoder.generate(input_ids=input_ids, File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1268, in generate self._validate_model_kwargs(model_kwargs.copy()) File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 964, in _validate_model_kwargs raise ValueError( ValueError: The followingmodel_kwargs are not used by the model: ['encoder_hidden_states', 'encoder_attention_mask'] (note: typos in the generate arguments will also show up in this list)

I have this error when i’m trying to run Dreambooth in Automatic1111 Webui.

First part of the error when I try to start a new entry with Dreambooth installed on SD. (But it starts normally, only this error appears) `Error loading script: api.py Traceback (most recent call last): File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py”, line 184, in load_scripts module = script_loading.load_module(scriptfile.path) File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/script_loading.py”, line 13, in load_module exec(compiled, module.dict) File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/api.py”, line 10, in <module> from extensions.sd_dreambooth_extension.dreambooth.sd_to_diff import extract_checkpoint File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/sd_to_diff.py”, line 36, in <module> from diffusers import ( ImportError: cannot import name ‘HeunDiscreteScheduler’ from ‘diffusers’ (/usr/local/lib/python3.8/dist-packages/diffusers/init.py)

Error loading script: main.py Traceback (most recent call last): File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py”, line 184, in load_scripts module = script_loading.load_module(scriptfile.path) File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/script_loading.py”, line 13, in load_module exec(compiled, module.dict) File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py”, line 5, in <module> from extensions.sd_dreambooth_extension.dreambooth.diff_to_sd import compile_checkpoint File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py”, line 214, in <module> def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]): TypeError: ‘type’ object is not subscriptable

LatentDiffusion: Running in v-prediction mode DiffusionWrapper has 865.91 M params. Loading weights [6bccbcc6] from /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Stable-diffusion/model.ckpt Applying xformers cross attention optimization. Model loaded. Loaded a total of 0 textual inversion embeddings. Embeddings: `

Then when I go into the GUI, it doesn’t show the Dreambooth tab.

[roperi]

@jvkap

EDIT: The following solved the ‘Dreambooth tab not showing in the web UI’ problem.

I solved the HeunDiscreteScheduler error by !pip install git+https://github.com/huggingface/diffusers. This should install a compatible version of Diffusers and fix the error.

But if then throws a transformers error do !pip install git+https://github.com/huggingface/transformers. This should install a compatible version of Transformers.

Having done that you should be left with one last error message :

def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]):
TypeError: 'type' object is not subscriptable

This seem to be a Python3.8 problem (i think doesn’t happen with Python 3.11). So open the file "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py" and then delete the type hint from the parameter of the function so change def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]): to def convert_text_enc_state_dict_v20(text_enc_dict). Also change def convert_text_enc_state_dict(text_enc_dict: dict[str, torch.Tensor]) to def convert_text_enc_state_dict(text_enc_dict):. These two deletions are 100% safe.

Save those two edited files, re-run everything and voila! You should finally get the Dreambooth tab.

PS. I’m using model 1.5 but this should work for 2.X

[d8ahazard]

Delete they typehint at 359 as well (->tuple…)

On Mon, Dec 19, 2022 at 9:41 PM mediocreatmybest @.***> wrote:

Just did an update to diffusers and transformers with: !pip install diffusers==0.10.2 transformers==4.25.1

Changed those two lines above as mentioned.

Looks like those two lines fixed the original colab with the extension.

Line 214 to:

def convert_text_enc_state_dict_v20(text_enc_dict):

Line 256 to:

def convert_text_enc_state_dict(text_enc_dict):

[image: dream] https://user-images.githubusercontent.com/80406625/208576918-8f9b7e3a-4914-41bf-a9ae-65438f213f20.png

Figured I should do a quick test run to see if any other errors.

Initializing dreambooth training… Patching transformers to fix kwargs errors. Traceback (most recent call last): File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/dreambooth.py”, line 361, in start_training from extensions.sd_dreambooth_extension.dreambooth.train_dreambooth import main File “/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py”, line 359, in <module> def main(args: DreamboothConfig, memory_record, use_subdir, lora_model=None, lora_alpha=1.0, lora_txt_alpha=1.0, custom_model_name=“”) -> tuple[ TypeError: ‘type’ object is not subscriptable Training completed, reloading SD Model.

This is with LORA Off, SD2.1 x768, Xformers, FP16, 8bitAdam enabled

— Reply to this email directly, view it on GitHub https://github.com/TheLastBen/fast-stable-diffusion/issues/914#issuecomment-1358797093, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMO4NHPFFPSUDF5346WOD3WOETFBANCNFSM6AAAAAASXLDMKI . You are receiving this because you were mentioned.Message ID: @.***>