9732b9c474
Use CRLF for newlines ( #18 )
...
* Update newlines to \r\n
* fix ndjson_stream newline change
---------
Co-authored-by: Foretack <69414142+Foretack@users.noreply.github.com>
Co-authored-by: boring_nick <boring-nick@users.noreply.github.com>
2023-11-14 20:59:59 +02:00
1171651732
temporarily disable log deletion
2023-11-10 18:19:33 +02:00
0ede6d3f23
fix: avoid duplicate status code
2023-10-25 12:08:38 +03:00
c44c9d8776
distinguish opted out channels and users
2023-10-24 18:49:53 +03:00
b160f8452e
Add contributing guide and setup files ( #16 )
...
* Add contributing guide and setup files
* set the default db url to localhost
* minor clarifications
* update docs on installing from source
---------
Co-authored-by: boring_nick <boring-nick@users.noreply.github.com>
2023-09-09 15:48:10 +03:00
b7a6bfbf7b
implement capabilities reporting
2023-09-03 21:24:32 +03:00
26084111f7
cleanup
2023-09-03 16:58:29 +03:00
c68e4c1112
split large channel log requests into multiple queries
2023-09-03 16:56:54 +03:00
0b0e45fe96
bump dependencies and clippy fixes
2023-08-30 22:37:48 +03:00
8f382b508a
Add query parameters to control date range of requested logs of channels and users. ( #15 )
...
* add from & to datetime params to channel logs query
* add from & to datetime params to user logs query
2023-08-30 22:05:43 +03:00
0af071874b
FailFish
2023-08-14 09:11:56 +03:00
58b66e6be4
feature: admins can now manually opt out users ( #12 )
...
* feature: admins can now manually opt out users
* fix: formatting
2023-07-30 17:32:32 +03:00
607cbfb8b9
return proper error when fetching latest logs of a user who opted out
2023-07-22 11:10:54 +03:00
d827f540e3
add api key info to admin openapi docs
2023-07-15 21:08:32 +03:00
7d2a72bcde
check optout on random line
2023-07-12 20:44:37 +03:00
3dd4fb17b6
make migrator errors more verbose
2023-07-09 22:00:29 +03:00
e8ed3654b7
redirect to latest available log instead of current date
2023-07-04 17:38:25 +03:00
70d8af69f6
fix admin check on optout
2023-06-29 17:04:54 +03:00
4098036d31
add offset support
2023-06-28 08:54:12 +03:00
c7be67cc71
generic migratable
2023-06-27 17:25:47 +03:00
672cdd93df
treat any query parameter value as true in bool params
2023-06-27 16:10:09 +03:00
e83fac75aa
fix delete query
2023-06-24 08:10:24 +03:00
e463c4dd90
set no cache header for metrics
2023-06-23 20:36:50 +03:00
a0c3a13383
implement cache-control header
2023-06-23 20:12:48 +03:00
901a2227a7
rework users cache to handle banned users
2023-06-23 19:22:16 +03:00
a4a6cf84d3
add compression layer
2023-06-23 09:02:13 +03:00
e05d44133e
add web metrics
2023-06-23 08:58:34 +03:00
16f4ba3e56
optimize getting available logs in the migrator, reduce spam
2023-06-23 08:49:34 +03:00
0d5283ab46
dont panic on migrations that take <1s
2023-06-23 08:47:26 +03:00
35ac15dd84
implement admin api
2023-06-22 20:56:26 +03:00
7e71464ddb
fix typo in migration name
2023-06-22 19:51:51 +03:00
1e4539e0dc
db migration to create projection for channel log dates
2023-06-22 16:58:58 +03:00
e382c62183
Revert "use distinct instead of group by for date list queries"
...
This reverts commit b50ff1c855
2023-06-22 16:53:30 +03:00
b50ff1c855
use distinct instead of group by for date list queries
2023-06-22 13:33:31 +03:00
5e986d2ac6
Revert "cache available channel log dates"
...
This reverts commit c2d34ff048
2023-06-22 13:31:47 +03:00
57e159d62e
get rid of unneded lifetime
2023-06-22 13:29:41 +03:00
959c9e91e9
rejoin channels once an hour
2023-06-21 19:35:13 +03:00
5099d27ec3
improve progress reporting
2023-06-20 18:49:28 +03:00
4e210f2be1
estimate migration progress
2023-06-20 18:27:12 +03:00
9dc18be434
bump dependencies
2023-06-19 21:28:34 +03:00
63652282b2
Merge pull request #9 from boring-nick/basic-json
...
Add jsonBasic logs query type
2023-06-19 18:34:05 +03:00
aa48e5f8f2
fix json stream and update web
2023-06-19 18:32:18 +03:00
42e04d57b5
Merge branch 'master' into basic-json
2023-06-19 18:16:22 +03:00
0f19298784
fix path
2023-06-19 14:22:41 +03:00
f3277c4c61
WIP basic json impl
2023-06-19 09:44:39 +03:00
e58197820d
increase retry count
2023-06-18 19:51:01 +03:00
52637f0b75
set default compression ratio to 5 after further testing
2023-06-18 19:42:35 +03:00
df5603bbe8
use doubledelta codec for timestamps to improve storage efficiency
2023-06-18 18:47:47 +03:00
dc5e09fda9
Free compression
...
After doing some testing, it looks like adding ZSTD to the timestamp and
user_id columns noticably improves compression ratio (14>17 on the test
dataset) without any real changes in performance.
Higher ZSTD levels do improve compression ratio slightly, but at the
cost of significantly slower compression (mainly a problem for migrating).
So 3 is kept as a reasonable compromise
2023-06-18 15:17:43 +03:00
1d2a3b5759
use axum state instead of extensions
2023-06-18 09:19:17 +03:00
boring-nick
boring_nick
bisspector