Beimingwu Docs

English

中文

English

中文

Appearance

How to Prepare a Learnware for Large Language Models ​

The learnware for large language models (LLMs) builds upon existing learnware standards while incorporating extensions tailored to the characteristics of LLMs. This document provides a detailed guide to the composition, implementation, and considerations for creating LLM learnwares.

Before proceeding, please read How to Prepare a Learnware? to familiarize yourself with the general learnware preparation process and requirements.

Basic Components of a Learnware ​

An LLM learnware is a zip file that must contain the following:

  • learnware.yaml: The configuration file describing the metadata of the learnware.
  • __init__.py: The core functionality implementation, including model loading and inference interfaces.
  • The statistical specification files: The filenames are customizable and recorded in learnware.yaml.
  • environment.yaml or requirements.txt: Specifies the runtime dependencies of the model.
  • weights_file_path/: A directory containing model weights and configuration files. For base models and fully fine-tuned models, this directory contains the complete model weights and configuration files. For parameter-efficient fine-tuning (PEFT) models, it contains the weights and configuration files for the fine-tuned parameters.

The weights_file_path/ directory typically includes the following files, which ensure accurate model loading and execution:

For base models and fully fine-tuned models:

  • config.json
  • generation_config.json
  • model.safetensors
  • tokenizer_config.json
  • tokenizer.json

For parameter-efficient fine-tuning models:

  • adapter_config.json
  • adapter_model.safetensors
  • tokenizer_config.json
  • tokenizer.json

The next sections detail the content and implementation of these files.

Model Invocation File __init__.py ​

The core functionality of the learnware is implemented in the __init__.py file. Below is a reference template for base models and fully fine-tuned models. The default directory for storing model weights and configuration files is weights. To facilitate the evaluation of the general capabilities of LLMs, the __init__.py file must provide a get_model interface that returns the instantiated model.

python
import os
import numpy as np
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from learnware.model import BaseModel


class MyModel(BaseModel):
    def __init__(self):
        super(MyModel, self).__init__(input_shape=None, output_shape=None)
        
        # Get the path to model weights and configuration files
        dir_path = os.path.dirname(os.path.abspath(__file__))
        model_path = os.path.join(dir_path, "weights")

        # Load the model and tokenizer
        self.model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", torch_dtype=torch.bfloat16)
        self.tokenizer = AutoTokenizer.from_pretrained(model_path)

        # Configure the tokenizer
        self.tokenizer.pad_token = self.tokenizer.eos_token
        self.max_length = 1024

    def fit(self, X: np.ndarray, y: np.ndarray):
        pass

    def predict(self, X):
        """
        Generates predictions based on input text.
        Args:
            X (list[str]): List of input texts.
        Returns:
            list[str]: Generated predictions.
        """
        inputs = self.tokenizer(X, padding=True, truncation=True, return_tensors="pt").to(self.model.device)
        outputs = self.model.generate(
            inputs["input_ids"],
            max_length=self.max_length,
            num_return_sequences=1,
            do_sample=True,
            temperature=0.7
        )
        return [self.tokenizer.decode(output, skip_special_tokens=True) for output in outputs]

    def finetune(self, X: np.ndarray, y: np.ndarray):
        pass

    def get_model(self):
        """
        Returns the model instance.
        Returns:
            torch.nn.Module: Model instance.
        """
        return self.model
import os
import numpy as np
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from learnware.model import BaseModel


class MyModel(BaseModel):
    def __init__(self):
        super(MyModel, self).__init__(input_shape=None, output_shape=None)
        
        # Get the path to model weights and configuration files
        dir_path = os.path.dirname(os.path.abspath(__file__))
        model_path = os.path.join(dir_path, "weights")

        # Load the model and tokenizer
        self.model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", torch_dtype=torch.bfloat16)
        self.tokenizer = AutoTokenizer.from_pretrained(model_path)

        # Configure the tokenizer
        self.tokenizer.pad_token = self.tokenizer.eos_token
        self.max_length = 1024

    def fit(self, X: np.ndarray, y: np.ndarray):
        pass

    def predict(self, X):
        """
        Generates predictions based on input text.
        Args:
            X (list[str]): List of input texts.
        Returns:
            list[str]: Generated predictions.
        """
        inputs = self.tokenizer(X, padding=True, truncation=True, return_tensors="pt").to(self.model.device)
        outputs = self.model.generate(
            inputs["input_ids"],
            max_length=self.max_length,
            num_return_sequences=1,
            do_sample=True,
            temperature=0.7
        )
        return [self.tokenizer.decode(output, skip_special_tokens=True) for output in outputs]

    def finetune(self, X: np.ndarray, y: np.ndarray):
        pass

    def get_model(self):
        """
        Returns the model instance.
        Returns:
            torch.nn.Module: Model instance.
        """
        return self.model

Below is a reference template for parameter-efficient fine-tuning learnware. The get_pretrained_path function returns the local path to the base model weights and configuration files based on the specified learnware ID in Beimingwu.

python
import os
import numpy as np
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from peft import get_peft_model, LoraConfig, TaskType
from learnware.model import BaseModel
from learnware.client import LearnwareClient


class MyModel(BaseModel):
    def __init__(self):
        super(MyModel, self).__init__(input_shape=None, output_shape=None)

        dir_path = os.path.dirname(os.path.abspath(__file__))
        adapter_path = os.path.join(dir_path, "adapter")

        # Get the path to the base model weights
        client = LearnwareClient()
        base_model_path = client.get_pretrained_path("Base Model ID")
        
        # Load the base model
        base_model = AutoModelForCausalLM.from_pretrained(base_model_path, device_map="auto", torch_dtype=torch.bfloat16)
        
        # Load the adapter model
        peft_config = LoraConfig.from_pretrained(adapter_path)
        self.model = get_peft_model(base_model, peft_config)
        self.model.load_adapter(adapter_path, adapter_name="lora")

        # Configure the tokenizer
        self.tokenizer = AutoTokenizer.from_pretrained(base_model_path)
        self.tokenizer.pad_token = self.tokenizer.eos_token
        self.max_length = 1024

    def fit(self, X: np.ndarray, y: np.ndarray):
        pass

    def predict(self, X):
        inputs = self.tokenizer(X, padding=True, truncation=True, return_tensors="pt").to(self.model.device)
        outputs = self.model.generate(
            inputs["input_ids"],
            max_length=self.max_length,
            num_return_sequences=1,
            do_sample=True,
            temperature=0.7
        )
        return [self.tokenizer.decode(output, skip_special_tokens=True) for output in outputs]

    def finetune(self, X: np.ndarray, y: np.ndarray):
        pass

    def get_model(self):
        return self.model
import os
import numpy as np
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from peft import get_peft_model, LoraConfig, TaskType
from learnware.model import BaseModel
from learnware.client import LearnwareClient


class MyModel(BaseModel):
    def __init__(self):
        super(MyModel, self).__init__(input_shape=None, output_shape=None)

        dir_path = os.path.dirname(os.path.abspath(__file__))
        adapter_path = os.path.join(dir_path, "adapter")

        # Get the path to the base model weights
        client = LearnwareClient()
        base_model_path = client.get_pretrained_path("Base Model ID")
        
        # Load the base model
        base_model = AutoModelForCausalLM.from_pretrained(base_model_path, device_map="auto", torch_dtype=torch.bfloat16)
        
        # Load the adapter model
        peft_config = LoraConfig.from_pretrained(adapter_path)
        self.model = get_peft_model(base_model, peft_config)
        self.model.load_adapter(adapter_path, adapter_name="lora")

        # Configure the tokenizer
        self.tokenizer = AutoTokenizer.from_pretrained(base_model_path)
        self.tokenizer.pad_token = self.tokenizer.eos_token
        self.max_length = 1024

    def fit(self, X: np.ndarray, y: np.ndarray):
        pass

    def predict(self, X):
        inputs = self.tokenizer(X, padding=True, truncation=True, return_tensors="pt").to(self.model.device)
        outputs = self.model.generate(
            inputs["input_ids"],
            max_length=self.max_length,
            num_return_sequences=1,
            do_sample=True,
            temperature=0.7
        )
        return [self.tokenizer.decode(output, skip_special_tokens=True) for output in outputs]

    def finetune(self, X: np.ndarray, y: np.ndarray):
        pass

    def get_model(self):
        return self.model

You can implement the interfaces in the MyModel class according to the specific needs of your model. In addition, we recommend that you instantiate the MyModel class after completing the __init__.py file and test the behavior of the interfaces to ensure that they function and behave as expected.

Learnware Statistical Specifications ​

For base model learnware, you do not need to provide a statistical specification. The system will automatically generate a general capability specification and store it in the package. For fully fine-tuned and parameter-efficient fine-tuned learnwares, you need to provide two types of specifications: Text RKME Specification and Generative Model Specification.

Examples:

  1. Text RKME Specification
python
from learnware.specification import generate_stat_spec

spec1 = generate_stat_spec(type="text", X=train_x)
spec1.save("rkme.json")
from learnware.specification import generate_stat_spec

spec1 = generate_stat_spec(type="text", X=train_x)
spec1.save("rkme.json")
  1. Generative Model Specification
py
from learnware.specification import generate_generative_model_spec

# you can provide `X` as `List[str]` type:
spec = generate_generative_model_spec(X=X)
# or you can provide `dataset` as `datasets.Dataset` type, where `dataset_text_field` is the name of the dataset text field:
spec = generate_generative_model_spec(dataset=train_dataset, dataset_text_field="text")
spec.save("generative.pth")
from learnware.specification import generate_generative_model_spec

# you can provide `X` as `List[str]` type:
spec = generate_generative_model_spec(X=X)
# or you can provide `dataset` as `datasets.Dataset` type, where `dataset_text_field` is the name of the dataset text field:
spec = generate_generative_model_spec(dataset=train_dataset, dataset_text_field="text")
spec.save("generative.pth")

Learnware Configuration File (learnware.yaml) ​

In addition to standard fields, LLM learnwares require the following additional fields:

  • weights_file_path: The relative path to model weights and configuration files, enabling the get_pretrained_path function to correctly locate the files.
  • required_learnware_ids: IDs of other learnwares that the model depends on, ensuring the proper downloading and loading of base or fine-tuned models.

Example learnware.yaml:

yaml
model:
  class_name: MyModel
  weights_file_path: weights  # New field: Relative path to model weights
  required_learnware_ids:  # New field: IDs of dependent learnware
    - base_model_id
  kwargs: {}
stat_specifications:
  - module_path: learnware.specification
    class_name: RKMETextSpecification
    file_name: rkme.json
    kwargs: {}
  - module_path: learnware.specification
    class_name: GenerativeModelSpecification
    file_name: generative.pth
    kwargs: {}
model:
  class_name: MyModel
  weights_file_path: weights  # New field: Relative path to model weights
  required_learnware_ids:  # New field: IDs of dependent learnware
    - base_model_id
  kwargs: {}
stat_specifications:
  - module_path: learnware.specification
    class_name: RKMETextSpecification
    file_name: rkme.json
    kwargs: {}
  - module_path: learnware.specification
    class_name: GenerativeModelSpecification
    file_name: generative.pth
    kwargs: {}

Model Runtime Dependencies ​

To ensure that the uploaded learnware is usable by others, you must explicitly specify the model’s runtime dependencies in the zip file. These requirements are consistent with those outlined in How to Prepare a Learnware?.