react-native-transformers

react-native-transformers is a React Native library for running Large Language Models (LLMs) from Hugging Face on your mobile applications locally. It supports both iOS and Android platforms, allowing you to leverage advanced AI models directly on your device without requiring an internet connection.

Features

• On-device transformer model support for both text generation and text embedding
• Local inference without internet connectivity
• Compatible with iOS and Android platforms
• Simple API for model loading and inference
• Support for Hugging Face models in ONNX format
• Built on top of ONNX Runtime for efficient model execution
• TypeScript support with full type definitions

Installation

To use react-native-transformers, you need to install onnxruntime-react-native as a peer dependency. Follow the steps below:

1. Install the peer dependency:

npm install onnxruntime-react-native
Copy

2. Install react-native-transformers:

npm install react-native-transformers
Copy

3. Configure React-Native or Expo

React Native CLI

• Link the onnxruntime-react-native library:

  npx react-native link onnxruntime-react-native
  Copy

Expo

• Install the Expo plugin configuration in app.json or app.config.js:

  {
    "expo": {
      "plugins": [
        "onnxruntime-react-native"
      ],
    }
  }
  Copy

4. Babel Configuration

You need to add the babel-plugin-transform-import-meta plugin to your Babel configuration (e.g., .babelrc or babel.config.js):

{
  "plugins": ["babel-plugin-transform-import-meta"]
}
Copy

Usage

Text Generation Example

import React from "react";
import { View, Text, Button } from "react-native";
import { Pipeline } from "react-native-transformers";

export default function App() {
  const [output, setOutput] = React.useState("");

  // Function to initialize the model
  const loadModel = async () => {
    await Pipeline.TextGeneration.init("Felladrin/onnx-Llama-160M-Chat-v1", "onnx/decoder_model_merged.onnx");
  };

  // Function to generate text
  const generateText = () => {
    Pipeline.TextGeneration.generate("Hello world", setOutput);
  };

  return (
    <View>
      <Button title="Load Model" onPress={loadModel} />
      <Button title="Generate Text" onPress={generateText} />
      <Text>Output: {output}</Text>
    </View>
  );
}
Copy

Text Embedding Example

import React from "react";
import { View, Text, Button } from "react-native";
import { Pipeline } from "react-native-transformers";

export default function App() {
  const [embedding, setEmbedding] = React.useState([]);

  // Function to initialize the model
  const loadModel = async () => {
    await Pipeline.TextEmbedding.init("Xenova/all-MiniLM-L6-v2");
  };

  // Function to generate embeddings
  const generateEmbedding = async () => {
    const result = await Pipeline.TextEmbedding.generate("Hello world");
    setEmbedding(result);
  };

  return (
    <View>
      <Button title="Load Model" onPress={loadModel} />
      <Button title="Generate Embedding" onPress={generateEmbedding} />
      <Text>Embedding Length: {embedding.length}</Text>
    </View>
  );
}
Copy

For detailed API documentation, please visit our TypeDoc documentation.

Contributing

Contributions are welcome! See the contributing guide to learn how to contribute to the repository and the development workflow.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgements

• ONNX Runtime for efficient model execution on mobile devices
• @xenova/transformers for transformer model implementations
• Hugging Face for providing pre-trained models and model hosting

External Links

• Expo Plugins Documentation
• ONNX Runtime Documentation
• Hugging Face Model Hub
• Babel Documentation

These links provide additional information on how to configure and utilize the various components used by react-native-transformers.