e58197820d
increase retry count
2023-06-18 19:51:01 +03:00
52637f0b75
set default compression ratio to 5 after further testing
2023-06-18 19:42:35 +03:00
df5603bbe8
use doubledelta codec for timestamps to improve storage efficiency
2023-06-18 18:47:47 +03:00
dc5e09fda9
Free compression
...
After doing some testing, it looks like adding ZSTD to the timestamp and
user_id columns noticably improves compression ratio (14>17 on the test
dataset) without any real changes in performance.
Higher ZSTD levels do improve compression ratio slightly, but at the
cost of significantly slower compression (mainly a problem for migrating).
So 3 is kept as a reasonable compromise
2023-06-18 15:17:43 +03:00
1d2a3b5759
use axum state instead of extensions
2023-06-18 09:19:17 +03:00
875aae5a61
improve openapi docs
2023-06-18 09:17:56 +03:00
552bb59022
better user ids/names cache
2023-06-17 20:31:16 +03:00
821d9acf64
log insert duration info
2023-06-17 09:26:49 +03:00
c2d34ff048
cache available channel log dates
2023-06-17 09:18:42 +03:00
51d04e2e49
make clickhouse flush interval configurable
2023-06-14 19:57:10 +03:00
1cb988f729
unescape tag values in json responses
2023-06-14 17:37:47 +03:00
36bd9f9904
minor streams cleanup
2023-06-14 08:58:21 +03:00
014c13132b
update docs
2023-06-14 08:44:19 +03:00
f199a67447
add support for ndjson log queries
2023-06-14 08:31:20 +03:00
8d9115364b
delete user logs on optout
2023-06-14 08:14:59 +03:00
f5e9f0e9cf
print warning about not being able to extract the user id tag
2023-06-13 20:12:28 +03:00
ccdac1346f
add config docs
2023-06-13 19:56:56 +03:00
34e8ee54ba
bump twitch-rs and print messages with errors
2023-06-13 19:45:35 +03:00
a0cc43540c
bump twitch-rs
2023-06-12 08:45:33 +03:00
ee3555e290
properly format text logs responses, respond with correct content-type
2023-06-11 16:45:23 +03:00
20d9e9cba1
optimize available logs query
2023-06-11 11:06:51 +03:00
e87a8e3ae3
bump twitch-rs
2023-06-11 10:46:56 +03:00
8db651c0b4
update web
2023-06-10 20:30:39 +03:00
ae5ed73298
actually functional optouts
2023-06-10 14:50:20 +03:00
70b8c37de7
add docs
2023-06-10 14:39:06 +03:00
6c30389ed5
migrator fixes
2023-06-10 11:06:14 +03:00
55b6a231fb
revert tags change
2023-06-09 23:36:46 +03:00
2507ea35c5
fix migrator tags whitelist
2023-06-09 23:35:31 +03:00
183b5d38d2
Create docker-publish.yml ( #8 )
...
* Create docker-publish.yml
* checkout submodules
* rename job
---------
Co-authored-by: boring_nick <boring-nick@users.noreply.github.com>
2023-06-09 23:07:32 +03:00
72edabd47b
fix user id extraction
2023-06-09 23:03:19 +03:00
7b48b83b42
Merge pull request #7 from boring-nick/clickhouse
...
Use clickhouse for storage
2023-06-09 22:38:55 +03:00
75e8668e61
Merge pull request #6 from boring-nick/streaming
...
More speed
2023-06-09 22:20:42 +03:00
c53c17a0ec
ignore roomstate
2023-06-09 22:20:16 +03:00
1abd028839
return 404 on empty logs
2023-06-09 22:14:08 +03:00
c95e3fe3cd
bump dependencies
2023-06-09 21:55:09 +03:00
6fac90c5b7
read day info in parallel
2023-06-09 21:52:41 +03:00
28a4fe7350
fix: use string lines
2023-06-09 21:34:39 +03:00
8a94c1e57c
make migrator slightly faster
2023-06-09 17:43:11 +03:00
77070c9349
even faster
2023-06-09 09:20:42 +03:00
d8eba3580e
optimize migrator
2023-06-09 09:04:59 +03:00
4f3dad9fd6
use single threaded parsing in migrator to spread out network load
2023-06-09 08:12:11 +03:00
1f433aacf7
clippy fixes
2023-06-09 08:07:40 +03:00
59918ab840
more speed
2023-06-08 21:26:06 +03:00
514142ecdd
streaming for json and text
2023-06-08 20:54:25 +03:00
27843dedda
initial streaming support
2023-06-08 09:52:54 +03:00
e20a9afe39
clippy fixes
2023-06-08 08:27:54 +03:00
0d01bb54ba
bump dependencies
2023-06-07 21:17:47 +03:00
1072071d89
use sync fs for getting available logs
2023-06-07 21:03:01 +03:00
9bf48c38e5
use new parser in migration
2023-06-07 19:29:04 +03:00
f7f1d12d29
report channel count in migration
2023-06-07 09:16:27 +03:00
boring_nick
boring-nick