Twitch logging service inspired by justlog
Go to file
Fijxu 3892d15c7e what? 2024-03-03 01:37:06 -03:00
.github/workflows multiarch docker builds 2023-12-13 22:50:16 +02:00
docs implement admin api 2023-06-22 20:56:26 +03:00
src NO OPTOUTS! 2024-03-03 01:35:20 -03:00
web@06e83668e0 what? 2024-03-03 01:37:06 -03:00
.dockerignore Revert "cache available channel log dates" 2023-06-22 13:31:47 +03:00
.gitignore add docs 2023-06-10 14:39:06 +03:00
.gitmodules NO OPTOUTS! 2024-03-03 01:35:20 -03:00
Cargo.lock bump dependencies and clippy fixes 2023-08-30 22:37:48 +03:00
Cargo.toml bump dependencies and clippy fixes 2023-08-30 22:37:48 +03:00
Dockerfile multiarch docker builds 2023-12-13 22:50:16 +02:00
README.md Add contributing guide and setup files (#16) 2023-09-09 15:48:10 +03:00
config.dist.json Add contributing guide and setup files (#16) 2023-09-09 15:48:10 +03:00
docker-compose.dev.yml Add contributing guide and setup files (#16) 2023-09-09 15:48:10 +03:00

README.md

Rustlog

Description

Rustlog is a Twitch logging service based on justlog. It provides the same web UI and API, but it utilizes Clickhouse for storage instead of text files.

Installation

Create a config.json file (see CONFIG.md)

Docker

version: "3.8"
  
services:
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    container_name: clickhouse
    volumes:
      - "./ch-data:/var/lib/clickhouse:rw"
    environment:
      CLICKHOUSE_DB: "rustlog"
    restart: unless-stopped
        
  rustlog:
    image: ghcr.io/boring-nick/rustlog:master
    container_name: rustlog
    ports:
      - 8025:8025 
    volumes:
      - "./config.json:/config.json"
    depends_on: 
      - clickhouse
    restart: unless-stopped

From source

  • Follow the Contributing excluding the last step
  • cargo build --release
  • The resulting binary will be at target/release/rustlog

Advantages over justlog

  • Significantly better storage efficiency (2x+ improvement) thanks to not duplicating log files and better compression (using ZSTD in Clickhouse)
  • Blazing fast log queries with response streaming and a highly performant IRC parser
  • Support for ndjson logs responses

Contributing

Requirements:

  • rust
  • yarn
  • docker with docker-compose (optional, will need to set up Clickhouse manually without it)

Steps:

  1. Clone the repository (make sure to include submodules!):
git clone --recursive https://github.com/boring-nick/rustlog

If you already cloned the repo without --recursive, you can initialize submodules with:

git submodule update --init --recursive
  1. Set up the database (Clickhouse):

This repository provides a docker-compose to quickly set up Clickhouse. You can use it with:

docker-compose -f docker-compose.dev.yml up -d

Alternatively, you can install Clickhouse manually using the official guide.

  1. Create a config file

Copy config.dist.json to config.json and configure your database and twitch credentials. If you installed Clickhouse with Docker, the default database configuration works.

  1. Build the frontend:
cd web
yarn install
yarn build
cd ..
  1. Build and run rustlog:
cargo run

You can now access rustlog at http://localhost:8025.

Migrating from justlog

See MIGRATION.md