GRPOConfig

The GRPOConfig class extends SFTConfig with additional parameters for Generative Reliability Policy Optimization (GRPO) reinforcement learning. GRPO is used to fine-tune models based on reward functions.

​

General Hyperparameters

​

GRPO Specific Hyperparameters

​

LoRA Specific Hyperparameters

​

SamplingParamsConfig

The SamplingParamsConfig class defines the parameters used for generating completions during the GRPO training process.

​

RewardFunctionsConfig

The RewardFunctionsConfig class defines the configuration for reward functions that evaluate and score model outputs during the GRPO training process. These reward functions are used to guide the model’s learning by providing feedback on the quality of generated completions.

​

RewardFunctionsRuntimeConfig

The RewardFunctionsRuntimeConfig class optionally defines the configuration for the runtime environment of reward functions during the GRPO training process.

​

Example Usage

from predibase import GRPOConfig, SamplingParamsConfig, RewardFunctionsConfig

def format_reward(prompt: str, completion: str, example: dict) -> float:
    """Validate output format"""

    try:
        if completion.startswith("```json"):
            return 1.0
        return 0.0
    except Exception:
        return 0.0

config = GRPOConfig(
    base_model="qwen3-8b",
    sampling_params=SamplingParamsConfig(
        max_tokens=512
    ),
    reward_fns=RewardFunctionsConfig(
        functions={
            "format": format_reward
        },
        # Optional: Additional packages to install for reward functions
        # if your reward functions require additional packages
        # runtime=RewardFunctionsRuntimeConfig(
        #     packages=[
        #         "mypkg",
        #     ]
        # ),
    )
)

​

Related Resources

We highly recommend reading the following resources to understand the GRPO reinforcement learning process before kicking off a GRPO job:

• Reinforcement Learning Documentation
• GRPO User Guide