project-lighter

/

ct_fm_feature_extractor

like 8

Follow

Lighter 7

Hi @fepegar ,

While we performed the pre-training on 24x128x128 with 3x1x1 spacing, we experimented with different patch sizes for the feature extractor when it came to search and retrieval. I would prioritize the needs of the use-case - i.e. if you're looking at specific local regions or larger FOVs. The only recommendation I would make is to keep the spacing to 3x1x1 (z,y,x) when possible.

Thank you, @surajpaib ! What would you use for the full FOV?

For the full FOV but whatever you get out of your data post resampling to 3x1x1 should work (followed this approach for the 3D-MIR datasets)

One consideration with full FOV would be that the final features might benefit from not just looking at avg pooled values. I'd suggest concat or such depending on what you might use these features for downstream.

Thank you!

Hello @fepegar ! Great package! Just to be clear, if we are just trying to run the feature_extractor (no additional fine-tuning) does the pre-training of 24x128x128 suggest that we only run the feature_extractor on no more than 24 slices?

In other words, to extract features should we use the code found in this file, where the patch_size is explicitly defined: https://github.com/project-lighter/CT-FM/blob/f58c89a75ff0270ba6e339027e40bf04a221735d/scripts/feature_extractor.py#L64C1-L78C46

as opposed to the example here: https://huggingface.co/project-lighter/ct_fm_feature_extractor

Just to be clear, I'm not the author of this.

Sorry! I meant to direct my question to @surajpaib - please let me know what you think when you can?

Hello @fepegar ! Great package! Just to be clear, if we are just trying to run the feature_extractor (no additional fine-tuning) does the pre-training of 24x128x128 suggest that we only run the feature_extractor on no more than 24 slices?

In other words, to extract features should we use the code found in this file, where the patch_size is explicitly defined: https://github.com/project-lighter/CT-FM/blob/f58c89a75ff0270ba6e339027e40bf04a221735d/scripts/feature_extractor.py#L64C1-L78C46

as opposed to the example here: https://huggingface.co/project-lighter/ct_fm_feature_extractor

Hi @meghutch

As I suggested previously, I'd recommend using a size that is appropriate for your use-case. If you are looking at tumors of a specific size dist. maybe the 24x128x128 doesn't work as well. As such, there is no default recommended size owing to the fact that we apply several crop and resize functions during the contrastive learning phase.

@meghutch If you'd be able to share more info about your use-case, I'm happy to guide further based on other examples we've tried for CT-FM