Multi-Modal Example Architecture
=================================

Last updated: 04/28/2025.

Introduction
------------

Now, verl has supported multi-modal training. You can use fsdp and 
vllm/sglang to start a multi-modal RL task. Megatron supports is also 
on the way.

Follow the steps below to quickly start a multi-modal RL task.

Step 1: Prepare dataset
-----------------------

.. code:: python

    # it will be saved in the $HOME/data/geo3k folder
    python examples/data_preprocess/geo3k.py

Step 2: Download Model
----------------------

.. code:: bash

    # download the model from huggingface
    python3 -c "import transformers; transformers.pipeline(model='Qwen/Qwen2.5-VL-7B-Instruct')"

Step 3: Perform GRPO training with multi-modal model on Geo3K Dataset
---------------------------------------------------------------------

.. code:: bash

    # run the task
    bash examples/grpo_trainer/run_qwen2_5_vl-7b.sh