Skip to content

LX-doctorAI1/DeltaNet

Repository files navigation

DeltaNet

简洁版

训练测试代码主入口:

main_basic.py: basic model

main_conditional.py: 使用一个condition的model

main_multi_conditional.py: 使用多个condition的model

所有代码运行所需参数已经在config文件中设置好

运行命令只需指定配置文件和所需使用的gpu id

示例: python main<model>.py --cfg config/iu_con_1.yml --gpu 0

详细版

Requirements

Data

Download IU and MIMIC-CXR datasets, and place them in data folder.

  • IU dataset from here
  • MIMIC-CXR dataset from here

Folder Structure

  • config : setup training arguments and data path
  • data : store IU and MIMIC dataset
  • Models: basic model and all our models
    • the layer define of our model
    • loss function
    • metrics
    • some utils
  • data_loader.py: dataloader
  • build_vocab.py: tokenizer
  • preprocessing: data preprocess
  • pycocoevalcap: Microsoft COCO Caption Evaluation Tools

Data preparation

  1. data

    • Download and unzip IU dataset to here
  2. preprocess_data.py

    • Create dataset split for IU dataset.
    • Preprocess report for normalizion, such as lower-case, replacing nonsense tokens(e.g., 'xxxx-a-xxxx').
    • Build vocabulary file.
  3. extract_feature.py

    • Extract visual feature for retrieve conditional image.
  4. retieve_conditional_pair.py

    • Retrieve conditional image using cosine similarity.

Training

  • dataset: iu / mimic
  1. main_<model>.py

    • The model will be trained using command python main_<model>.py --cfg config/<dataset_N> --expe_name <experiment name> --gpu <GPU_ID>
    • More options can be found in config/opts.py file.
  2. Models

    • Basic.py: Basic Model
    • Conditional.py: DeltaNet condition generation model
    • MultiConditional.py: DeltaNet multiple conditional generation model
    • Generator.py and Beam.py: Generate report using beam search
    • Modules.py: Implement of scaled dot-product attention
    • SubLayers.py: Implement of multi-head attention
    • misc.py and utils.py: Utils function

Testing

test.py: Generate report from trained model using command python test.py --pretrained <path to the checkpoint file>

Evaluate

  1. metrics.py

    • Evaluate the generated report.
  2. pycocoevalcap

    • Microsoft COCO Caption Evaluation Tools
    • Need clone from GitHub and modified the code to work with Python 3

Cite

Xian Wu, Shuxin Yang, Zhaopeng Qiu, Shen Ge, Yangtian Yan, Xingwang Wu, S.Kevin Zhou, Li Xiao. DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis. Coling, 2022

If you have any problem with the code, please contact Xian Wu(kevinxwu@tencent.com), Shuxin Yang(aspenstarss@gmail.com) or Li Xiao(andrew.lxiao@gmail.com).

Thanks

Joseph Paul Cohen, Joseph D. Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, Hadrien Bertrand. TorchXRayVision: A library of chest X-ray datasets and models. Medical Imaging with Deep Learning. https://github.com/mlmed/torchxrayvision, 2020

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages