accuracy problem at the late fusion step

Question

During reproducing the late fusion step on the Aachen Day-Night v1.1 data set(with 4 global features that kapture provided, AP-GeM-LM18/ DELG/ densevlad_multi/ netvlad_vd16pitts/)，I found the accuracy at night is very low compared to the results in the paper.

with gharm Top20 config2, I got

No.	day	night
mine	90.7 / 97.1 / 99.5	68.6 / 83.8 / 95.8
your paper	90.5 / 96.8 / 99.4	74.9 / 90.1 / 98.4
diff	+0.2 / +0.3 / +0.1	-6.3 / - 6.3 / - 2.6

I wonder whether it‘s an implementation problem or is there any other trick?

What did I do：

Step1: use full dataset to construct a single map according to https://github.com/naver/kapture-localization/blob/main/pipeline/examples/run_aachen-v11.sh

Step2: do late fusion using https://github.com/naver/kapture-localization/blob/823f85430c4739b398b5a1cf11ef7d942b0e917d/tools/kapture_image_retrieval_late_fusion.py Since there is no example of fusion script, I wrote one based on my understanding:

`# 0a) Define paths and params PYTHONBIN=python3.8 WORKING_DIR=${PWD} DATASETS_PATH=${WORKING_DIR}/datasets DATASET=Aachen-Day-Night-v1.1 mkdir -p ${DATASETS_PATH}

TOPK=20 # number of retrieved images for mapping and localization KPTS=20000 # number of local features to extract

#-gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_features \

#-gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18 ${WORKING_DIR}/${DATASET}/global_features/DELG ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts \

#1)kapture_image_retrieval_late_fusion kapture_image_retrieval_late_fusion.py -v debug
-i ${WORKING_DIR}/${DATASET}/map_plus_query/
–query ${WORKING_DIR}/${DATASET}/query/
-o ${WORKING_DIR}/${DATASET}/pairs_fusion.txt
–topk ${TOPK}
‘generalized_harmonic_mean’ --weights 1 1 1 1`

Step3: using the pairs_fusion.txt obtained in step2, I did: `# 0a) Define paths and params PYTHONBIN=python3.8 WORKING_DIR=${PWD} DATASETS_PATH=${WORKING_DIR}/datasets DATASET=Aachen-Day-Night-v1.1 mkdir -p ${DATASETS_PATH}

TOPK=20 # number of retrieved images for mapping and localization KPTS=20000 # number of local features to extract

#7) localization pipeline LOCAL=r2d2_WASF_N8_big GLOBAL=Fusion kapture_pipeline_localize.py -v debug -f
-s compute_image_pairs compute_matches geometric_verification
-i ${WORKING_DIR}/${DATASET}/mapping
–query ${WORKING_DIR}/${DATASET}/query
-kpt ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/keypoints
-desc ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/descriptors
–pairsfile-path ${WORKING_DIR}/${DATASET}/pairs_fusion.txt
-matches ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_no_gv/matches
-matches-gv ${WORKING_DIR}/${DATASET}/local_features/${LOCAL}/NN_colmap_gv/matches
–colmap-map ${WORKING_DIR}/${DATASET}/colmap-sfm/${LOCAL}/Resnet101-AP-GeM-LM18
-o ${WORKING_DIR}/${DATASET}/colmap-localize/${LOCAL}/${GLOBAL}
–topk ${TOPK}
–config 2 `

additional questions:

1.how to correctly use the ‘kapture_image_retrieval_late_fusion.py’ in step2? I’m very confused especially with the -gfeat parameter; As shown above, what I passed to gfeat can’t work properly so I use the default

Thanks in advance

[yocabon]

Hi, I’ll answer the additional question since it’s probably related to the low results you got. In kapture_image_retrieval_late_fusion.py, -i is the mapping data; not the map_plus_query data.

so in order to make that work, you would use

kapture_create_kapture_proxy.py with

-i ${WORKING_DIR}/${DATASET}/mapping -o ${WORKING_DIR}/${DATASET}/gharm/mapping -gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_features

(the resulting ${WORKING_DIR}/${DATASET}/gharm/mapping/reconstruction/global_features should have 4 folders)

and

kapture_create_kapture_proxy.py with

-i ${WORKING_DIR}/${DATASET}/query -o ${WORKING_DIR}/${DATASET}/gharm/query -gfeat ${WORKING_DIR}/${DATASET}/global_features/AP-GeM-LM18/global_features ${WORKING_DIR}/${DATASET}/global_features/DELG/global_features ${WORKING_DIR}/${DATASET}/global_features/densevlad_multi/global_features ${WORKING_DIR}/${DATASET}/global_features/netvlad_vd16pitts/global_features

then if you want gharm pairs for mapping (map <-> map – which we would do unless we built a map with distance or many pairs), you would call

kapture_image_retrieval_late_fusion.py -v debug \
-i ${WORKING_DIR}/${DATASET}/gharm/mapping \
-o ${WORKING_DIR}/${DATASET}/pairs_fusion_mapping.txt \
--topk ${TOPK} generalized_harmonic_mean

and for query -> map pairs

kapture_image_retrieval_late_fusion.py -v debug \
-i ${WORKING_DIR}/${DATASET}/gharm/mapping \
--query ${WORKING_DIR}/${DATASET}/gharm/query \
-o ${WORKING_DIR}/${DATASET}/pairs_fusion_query.txt \
--topk ${TOPK} generalized_harmonic_mean

note that the -gfeat parameter is only here to help you select the global features types you want to use, in this case you would do -gfeat AP-GeM-LM18 DELG netvlad_vd16pitts for example if you didn’t want to use netvlad.

I hope this helps.

[humenbergerm]

That’s really strange and it never happened to us. Are you sure that you use a different output folder for each run or that you delete the last results before a new run?