Rule Engine with Abstract Syntax Tree (AST)

Rohan Jaiswal

Completed work

Backend Engineer

Fullstack Engineer

Docker

FastAPI

Next.js

Web Development

Rule Engine with Abstract Syntax Tree (AST)

Project Overview

A rule engine with an Abstract Syntax Tree (AST) is a system that parses, represents, and evaluates rules (Expressions) using a hierarchical tree structure. In this context, the AST serves as an intermediary between the textual representation of a rule and its execution, enabling dynamic and flexible decision-making processes. Enabling system to dynamically create, combine, and modify these rules.

Features

Simple and Clean UI

Represent rules using Abstract Syntax Trees (AST)

tokens -> lexical analyzer -> parser -> evaluater (interpreter)

Create, combine, modify, evaluate, and delete complex rules

Comprehensive error handling and input validation

Catalog is implemented for type validation and user ease

RESTful API for rule management and evaluation

System Architecture

The application follows a 3-tier architecture:

Frontend: A React-based user interface

Backend API: FastAPI-based RESTful API for rule management

Database: PostgreSQL for persistent storage of rules

Getting Started

Prerequisites

Python 3.8+

Node.js 14+

PostgreSQL

Docker

Installation

Clone the repository:

git clone https://github.com/deadrohan19/rule_engine_with_ast.git
cd rule_engine_ast_with_ast

Set up a virtual environment for the backend:

python -m venv venv
source venv/bin/activate  # On Windows, use `venv\Scripts\activate`

Install dependencies If poetry isn't previously installed, install it first. Then continue to create a venv, and install dependencies

Set up environment variables: Create a .env file in the project root with the following content:

DB_USER=your_db_user
DB_PASSWORD=your_db_password
DB_HOST=localhost
DB_PORT=5432
DB_NAME=rule_engine_db

Setup postgres in docker (Docker Setup)

Initialize the database:

python -m rule_engine.models

Install frontend dependencies:

cd frontend
npm install

Usage

Running the Server

To start the FastAPI development server:

python main.py --dev

By default, the server runs on http://0.0.0.0:5000. You can specify a different host and port using the --host and --port arguments.

Running the Frontend

To start the React development server:

cd frontend
npm run dev

The frontend will be available at http://localhost:3000.

API Endpoints

POST /create_rule: Create a new rule

POST /evaluate_rule: Evaluate a rule against provided data

POST /combine_rules: Combine multiple rules

GET /get_catalog: Retrieve the attribute catalog

GET /get_rule: Retrieve a specific rule

GET /get_all_rule_names: Retrieve all rule names

DELETE /delete_rule: Delete a specific rule

For detailed API documentation, run the server and visit http://localhost:5000/docs.

Design Choices

Abstract Syntax Tree (AST): ASTs used to represent rules, allowing for efficient evaluation and easy modification of complex rule structures.

FastAPI Framework: Chosen for its high performance, automatic API documentation, and type checking capabilities.

PostgreSQL: Used for persistent storage of rules, offering reliability and support for complex queries.

Pydantic Models: Employed for request/response validation and serialization.

SQLAlchemy ORM: Provides a high-level abstraction for database operations, making it easier to work with different database systems.

Error Handling: Custom exception classes for different types of errors (SyntaxError, InvalidTokenError, TypeError, InsufficientDataError) to provide clear and specific error messages.

Modular Structure: The project is organized into separate modules (parser, abstract_tree, models, etc.) for better maintainability and separation of concerns.

React Frontend: Utilizes React for a dynamic and responsive user interface, with components for rule creation, visualization, and management.

String to AST

Convert to tokens

Lexer would give tokens a type

On the basis of the token types, according to above grammar Parser would create AST

If syntax is not matching to grammar, which implies that syntax of received rule is invalid

At last, We can use AST to evaluate for any data, it is true or not.

As the rule is in tree form, then we can easily modify it in any way, by counting the number of left and rights taken to reach modification juncture.

Testing

To run the test suite:

python main.py --tests

This will execute all unit tests using pytest.

Dependencies

Backend:

FastAPI: Web framework for building APIs

Uvicorn: ASGI server for running FastAPI

SQLAlchemy: SQL toolkit and ORM

Pydantic: Data validation and settings management

Psycopg2: PostgreSQL adapter for Python

Python-dotenv: Loading environment variables from .env files

Pytest: Testing framework

Frontend:

React: JavaScript library for building user interfaces

Next.js: React framework for production-grade applications

Axios: Promise-based HTTP client for making API requests

React-d3-tree: Library for rendering tree diagrams

Tailwind CSS: Utility-first CSS framework

For a complete list of dependencies with version numbers, refer to the pyproject.toml file for the backend and package.json for the frontend.

Frontend Components

The frontend consists of three main components:

page.tsx: The main page component that renders the rule engine interface. It includes tabs for creating, combining, and evaluating rules, as well as displaying the rule tree.

route.ts: Contains API route definitions and functions for making requests to the backend server.

rule-tree.tsx: A component for rendering the Abstract Syntax Tree visualization of rules using react-d3-tree.

These components work together to provide a user-friendly interface for managing and visualizing rules in the rule engine.

create rule

evaluate rule

Docker Setup

To run the PostgreSQL database in a Docker container:

Make sure Docker is installed on your system.

Pull the PostgreSQL image:

docker pull postgres

Run the PostgreSQL container:

docker run --name rule-engine-db -e POSTGRES_PASSWORD=your_password -e POSTGRES_DB=rule_engine_db -p 5432:5432 -d postgres

Update the .env file with the appropriate Docker container details:

DB_USER=postgres
DB_PASSWORD=your_password
DB_HOST=localhost
DB_PORT=5432
DB_NAME=rule_engine_db

This setup allows you to run the PostgreSQL database in a containerized environment, providing isolation and easy setup across different development environments.

References

Let's Build a Simple Interpreter by Ruslan Spivak

React-d3-tree

Shadcn/ui

To Be Implemented

Auto extend the input area when content exceed current viewport

In evaluate method UI,

Add option to directly add - {attributes}: {value} like in table form

Extend to add more datatypes

This README provides a comprehensive guide to setting up, running, and understanding the Rule Engine project, including both backend and frontend components. For any additional questions or issues, please open an issue in the GitHub repository.

Like this project

Completed work

Posted Feb 27, 2025

Rule Engine where we can create and evaluate rules with the help of Abstract Syntax Tree and have interactive visualization

Likes

Views

Rule Engine with Abstract Syntax Tree (AST)

Rule Engine with Abstract Syntax Tree (AST)

Project Overview

Table of Contents

Features

System Architecture

Getting Started

Prerequisites

Installation

Usage

Running the Server

Running the Frontend

API Endpoints

Design Choices

String to AST

Testing

Dependencies

Frontend Components

create rule

evaluate rule

Docker Setup

References

To Be Implemented