Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Compressed Graph doesn't reflect merged quantizers

See original GitHub issue

Reproduce Issue:

Run resnet50_imagenet_int8.json with target_device: "VPU"

Run following snippet after create_compressed_model.

quantize_nxnodes = []
g = model.get_graph()
for nxnode, node_dict in g._nx_graph.nodes.items():
    if 'symmetric_quantize' in nxnode:
        quantize_nxnodes.append(nxnode)
print("#Quantize nodes in graph: {}".format(len(quantize_nxnodes)))
print("#QuantizeId: {}".format(len(compression_ctrl.all_quantizations.items())))

Output:

#Quantize nodes in graph: 125
#QuantizeId: 121

The redundant quantize nodes are 25, 80, 152, 258.

resnet50-vpu-compressed_graph

Issue Analytics

State:
Created 3 years ago
Comments:8 (3 by maintainers)

Top GitHub Comments

1reaction

vshamporcommented, Dec 7, 2020

@vuiseng9 the operations graph does not store this information, but the controller does. Check the NonWeightQuantizerInfo structs as contained in QuantizationController.non_weight_quantizers, namely NonWeightQuantizerInfo.affected_ia_op_exec_contexts. This list will store all node addresses corresponding to a single quantizer module. Probably should add a method to the controller that queries an insertion location in the graph and returns the actual quantizer module reference.

0reactions

vuiseng9commented, Dec 7, 2020

Thanks @vshampor @ljaljushkin.

For completeness, attaching a sample to access the information as you suggested.

g = model.get_graph()

for nwq_id, nwq_info in compression_ctrl.non_weight_quantizers.items():
    if len(nwq_info.affected_ia_op_exec_contexts) > 1:
        print("\nShared Non-Weight QuantizerId: {}".format(str(nwq_id)))
        for ia_op_exc_ctx in nwq_info.affected_ia_op_exec_contexts:
            print("\tAffected Node: {}".format(g.get_node_id_by_iap_context(ia_op_exc_ctx)))

Shared Non-Weight QuantizerId: ResNet/Sequential[layer1]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm_0
	Affected Node: 20 ResNet/Sequential[layer1]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm
	Affected Node: 24 ResNet/Sequential[layer1]/Bottleneck[0]/Sequential[downsample]/BatchNorm2d[1]/batch_norm

Shared Non-Weight QuantizerId: ResNet/Sequential[layer2]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm_0
	Affected Node: 75 ResNet/Sequential[layer2]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm
	Affected Node: 79 ResNet/Sequential[layer2]/Bottleneck[0]/Sequential[downsample]/BatchNorm2d[1]/batch_norm

Shared Non-Weight QuantizerId: ResNet/Sequential[layer3]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm_0
	Affected Node: 147 ResNet/Sequential[layer3]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm
	Affected Node: 151 ResNet/Sequential[layer3]/Bottleneck[0]/Sequential[downsample]/BatchNorm2d[1]/batch_norm

Shared Non-Weight QuantizerId: ResNet/Sequential[layer4]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm_0
	Affected Node: 253 ResNet/Sequential[layer4]/Bottleneck[0]/BatchNorm2d[bn3]/batch_norm
	Affected Node: 257 ResNet/Sequential[layer4]/Bottleneck[0]/Sequential[downsample]/BatchNorm2d[1]/batch_norm

Top Results From Across the Web

Quantisation - an overview | ScienceDirect Topics

Quantization is defined as a lossy data compression technique by which intervals of data are grouped or binned into a single value (or...

A Survey of Quantization Methods for Efficient Neural Network ...

research, which mostly focused on finding compression methods that would not change the signal too ... Figure 1: Comparison between uniform quantization.

Image Compression and the Discrete Cosine Transform

The DCT works by separating images into parts of differing frequencies. During a step called quantization, where part of compression actually occurs, the...

Deep Task-Based Quantization - PMC - NCBI

The operation of the scalar ADCs is modeled as an intermediate activation layer. Unlike previous works which combined fixed uniform quantizers as part...

Analyzing and Predicting Large Vector-, Graph - mediaTUM

data” does not only come in sheer volume, it is about the complexity of the ... Figure 1.4: Minimum Description Length: Compressing the...