Commit graph

326 commits

Author SHA1 Message Date
jj 3d95361c09
api/cookie: validate cookie file format 2024-11-26 13:51:49 +00:00
jj 31d65c9fb7
api/cookie: validate service names for cookies 2024-11-26 13:44:51 +00:00
wukko d4bcb1ba61
api/service-config: add new domains for vk 2024-11-26 18:21:44 +06:00
wukko e2f0123418
api/tests/tiktok: add an age restricted video test 2024-11-24 19:26:59 +06:00
wukko a1fa79f2f5
api/tikok: catch an age restriction error 2024-11-24 19:26:44 +06:00
wukko 2433681d8b
api/package: bump version to 10.4.1 2024-11-24 19:08:40 +06:00
wukko 8a24dbb42d
api/match-action: fix audio in tiktok picker
it didn't have an audio format in the filename, so it either failed or downloaded without an extension.

closes #870
2024-11-24 19:02:10 +06:00
wukko cdd349cfb6
api/tests/rutube: add a region locked video test 2024-11-24 18:44:07 +06:00
wukko 6039eae6a3
api/rutube: catch a region lock error
closes #930
2024-11-24 18:43:50 +06:00
wukko 9b0e4ab0bd
api/tests/soundcloud: add tests for region locked and paid songs 2024-11-24 18:35:32 +06:00
wukko 43c3294230
api/soundcloud: catch region locked and paid songs and show an error 2024-11-24 18:35:07 +06:00
wukko eb52ab2be8
api/vimeo: return accidentally remove merge function 2024-11-24 18:19:56 +06:00
wukko 1cbffc2d75
api/stream/types: convert metadata in one place
also sanitize values & throw an error if tag isn't supported
2024-11-24 18:13:22 +06:00
wukko 6770738116
api/create-filename: build & sanitize filenames in one place 2024-11-24 18:12:21 +06:00
wukko 407c27ed86
api/utils: rename metadata converter function 2024-11-24 14:55:46 +06:00
wukko 6a430545d2
api/utils/cleanString: add more forbidden chars 2024-11-24 14:55:10 +06:00
wukko 6bb412852d
api/package: bump version to 10.4
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-11-24 00:37:52 +06:00
wukko ff9e248e4f
api/util/test: add twitter to finnicky list
they seemingly blocked ips of github workers
2024-11-23 15:42:47 +06:00
wukko 5b445d5c7e
api/youtube: catch even more innertube errors 2024-11-23 15:37:42 +06:00
jj b31c126cec
api/instagram: fix module not using graphql api 2024-11-18 17:34:48 +00:00
wukko 26e051fcd8
api/package: bump version to 10.3.3
Some checks failed
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2024-11-16 22:29:32 +06:00
wukko 606f0fd29a
api/stream/internal: workaround for wrong bsky content-type, refactor 2024-11-16 22:15:13 +06:00
wukko b61b8c82a2
api/bluesky: use hls video cdn directly 2024-11-16 21:57:14 +06:00
wukko 09c66fead0
api/package: bump version to 10.3.2
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-11-15 20:35:06 +06:00
wukko 3de3e9e158
api: remove support for vine cuz the archive is dead
masterful gambit elon musk
2024-11-15 18:29:21 +06:00
jj f7dc6cebad
all: add space after catch 2024-11-15 12:19:49 +00:00
jj 4c006b2291
api/test: add vk to finnicky services 2024-11-15 12:11:29 +00:00
jj cf40f0542f
api/test: make deepsource happy 2024-11-13 17:27:26 +00:00
jj f6bffe543c
api/test: replace test.js with test-ci.js 2024-11-13 17:26:15 +00:00
jj 91e8ef8ab4
api/test-ci: add functionality for running all tests 2024-11-13 17:26:15 +00:00
jj aaf7077364
api/test: split up tests into individual files 2024-11-13 17:26:15 +00:00
KwiatekMiki 0e09bf9895
api/service-config: recognize facebook's mobile subdomain (#891) 2024-11-13 22:35:45 +06:00
wukko 3fe2bd3b7c
api/youtube: add missing else to adaptive codec fallback 2024-11-13 22:23:45 +06:00
wukko 225a721805
api/tests: allow vk tests to fail 2024-11-13 18:48:36 +06:00
wukko dec977e34d
api/youtube: fix variable shadowing in normalizeQuality 2024-11-13 18:45:18 +06:00
wukko c88e21d4a8
api/youtube/adaptive: refactor, avoid extra loops, fallback all codecs 2024-11-13 18:41:57 +06:00
wukko 5f1c19d0f1
api/youtube: add no matching format error
this error is returned when cobalt got a response from innertube, but couldn't find a matching combo of video and audio streams. sometimes youtube returns only video or only audio per format combo for whatever reason.
2024-11-13 15:00:09 +06:00
wukko 8b972c7a85
api/youtube: disable hls if user prefers av1 2024-11-13 14:50:13 +06:00
wukko b6e827c6f9
api/youtube: improve video quality normalization once again 2024-11-13 14:49:51 +06:00
wukko 8fc9ca2916
api/bluesky: add a dispatcher & update unknown error message
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run service tests / test service functionality (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-11-11 12:23:53 +06:00
wukko f50bd6339b
api/service-config: add support for loom embed links
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
Run service tests / test service functionality (push) Has been cancelled
Run tests / check lockfile correctness (push) Has been cancelled
Run tests / web sanity check (push) Has been cancelled
Run tests / api sanity check (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2024-11-07 20:53:25 +06:00
jj 5a7635cdf7
api/cookie: write cookies only if from-file cookie was changed
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
Run service tests / test service functionality (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
2024-11-02 18:48:26 +00:00
jj c44a5ecc89
api/cookie: fix cookie.set() being ran only once 2024-11-02 18:46:56 +00:00
wukko ca665c5382
api: replace psl with homegrown & up-to-date fork
finally no more punycode warning
2024-11-02 21:19:19 +06:00
wukko 37517875db
api/package: update dependencies 2024-11-02 20:18:59 +06:00
jj e2b4141fc7
api/memory-store: unref timeout so it doesn't hold up process 2024-11-02 11:33:21 +00:00
wukko ab3af731e7
api/package: bump version to 10.3 2024-11-02 17:19:28 +06:00
jj cba308aabd
api/test-ci: reduce stream lifespan
the streams have picked up smoking
2024-11-02 11:13:04 +00:00
jj 2f89f79b14
api/memory-store: ; 2024-11-02 11:12:39 +00:00
wukko 44e08e8474
api/config: separate error if statements for session & instance count 2024-11-02 16:48:34 +06:00
wukko 541bf04575
api/services: fix createStream calls in pickers
oops
2024-11-02 16:43:36 +06:00
jj 382873dc11
api/core: fix main cluster being unable to handle itunnels 2024-11-02 09:59:48 +00:00
jj 5a66af514e
api: make deepsource happy 2024-11-01 17:24:22 +00:00
jj 90d57ab6ea
api/config: store tunnelPort in env 2024-11-01 17:02:29 +00:00
jj d48cc8fc07
api/cookie: implement cluster synchronization 2024-11-01 16:43:01 +00:00
jj 42ec28a642
api/cookie: update cookies value-by-value in manager 2024-11-01 14:58:04 +00:00
jj f098da870c
api/cookie: pick cookie at random instead of round-robin 2024-11-01 14:55:00 +00:00
jj 1c78dac7ed
api/cluster: implement broadcast helper 2024-11-01 14:49:52 +00:00
jj 2351cf74f4
api/cookie: formatting 2024-11-01 14:05:18 +00:00
jj 48883486fa
api/api-keys: load keys once per cluster 2024-11-01 13:57:53 +00:00
jj 3f505f6520
api: wait for cluster to finish preparing 2024-11-01 13:30:32 +00:00
jj 2317da5ba5
api: add support for redis to ratelimiter cache 2024-11-01 13:26:18 +00:00
jj d466f8a4af
api: upgrade express-rate-limit to v7, reuse key generator 2024-11-01 12:54:16 +00:00
jj 693204b799
api/store: use basic strings instead of hashes for keys 2024-11-01 12:20:01 +00:00
jj 66cb8d360d
api: move hmac secrets to single file 2024-11-01 12:16:53 +00:00
jj 40d6a02b61
api: cluster support
still missing synchronization for some structures
2024-10-31 22:59:06 +00:00
jj 2d6d406f48
api/crypto: use buffers for salt directly instead of hex strings 2024-10-31 22:42:46 +00:00
jj 93e6344fc7
api/stream/manage: make itunnel port configurable
this allows us to bind internal streams to
a specific worker in the future
2024-10-31 22:35:26 +00:00
jj 132255b004
api/stream/manage: use cobalt Store for stream cache 2024-10-31 22:33:32 +00:00
jj 11314fb8d1
api/store: implement has() method 2024-10-30 19:21:45 +00:00
jj 18acad19b9
api: implement redis/memory store for cache 2024-10-30 19:06:46 +00:00
jj 5e92b649a3
api: add API_REDIS_URL env 2024-10-30 18:59:20 +00:00
wukko 2c451c69d0
api/youtube: rename quality variable in matchQuality
Some checks failed
Run service tests / test service functionality (push) Has been cancelled
Run service tests / test service: ${{ matrix.service }} (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-10-31 21:43:02 +06:00
wukko 3dd6165472
api/youtube: slight refactor of matchQuality 2024-10-31 21:37:11 +06:00
wukko 5470926d52
api/youtube: adjust matched resolution
heights like 714 are now adjusted to 720, so that preferred quality is picked correctly
2024-10-31 21:31:39 +06:00
wukko da72b9615e
api/youtube: use best quality if all else fails 2024-10-31 21:18:34 +06:00
wukko 98acea6c58
api: bump version to 10.2.1
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-10-31 00:14:26 +06:00
wukko 776c4f4dba
api/stream/manage: don't use clones in node cache 2024-10-30 23:56:14 +06:00
wukko 406ac7613c
api/youtube: make sure language exists when checking for hls dubs
oops
2024-10-30 22:55:50 +06:00
wukko 44f842997e
api & web: bump version to 10.2 2024-10-30 18:29:53 +06:00
wukko 367cab0de4
api/youtube: update hls vp9 container to webm
way better compatibility this way
2024-10-30 17:18:18 +06:00
wukko f610058b82
api/stream/types/merge: encode audio to aac or opus if hls
audio is encoded to opus only if it's a youtube hls stream with webm container
2024-10-30 17:18:18 +06:00
wukko e081751c59
api/youtube: fix dubbed audio track matching 2024-10-28 23:05:56 +06:00
wukko 17c020fe22
api/youtube: fix dubbed hls audio marking 2024-10-28 21:38:25 +06:00
wukko 0b4d703d0f
api/utils: remove unused functions 2024-10-28 19:56:37 +06:00
wukko cdfc91844d
api/schema: update youtubeDubLang to accept all valid language codes 2024-10-28 19:56:18 +06:00
wukko b14c618228
api/youtube: pick a default track for videos with ai dubs 2024-10-28 19:35:08 +06:00
wukko 14ca47b73d
api/youtube: make mp3 the best format for hls audio 2024-10-28 18:30:01 +06:00
jj 53e6085095
api/stream: don't override content-length for hls transform 2024-10-28 11:55:15 +00:00
wukko 6b1eadbe09
api/util/tests: add youtube hls tests 2024-10-28 16:59:50 +06:00
wukko 866427a7a7
api/youtube: fix local variable overlap 2024-10-28 16:55:44 +06:00
wukko effec1bfb9
api/youtube: return correct audio url in hls mode
my disappointment in its quality is immeasurable
2024-10-28 16:45:48 +06:00
wukko 0ddb3e3ecc
api/match-action: add isHLS to audio stream info 2024-10-28 16:45:30 +06:00
wukko fba6ba09c2
api/youtube: add hls codec fallback, update hls error code, refactor
also fixed best audio format
2024-10-28 15:45:18 +06:00
wukko c9eefc4d55
api/youtube: add an option to use HLS streams
- added `youtubeHLS` variable to api
- added youtube HLS parsing & handling
2024-10-28 15:17:54 +06:00
wukko 24ae08b105
api/stream: add isHLS to stream cache 2024-10-28 15:15:41 +06:00
wukko a46e04358a
api/match-action: rename isM3U8 to isHLS and u to url 2024-10-28 15:14:36 +06:00
wukko 7c516c0468
api/cookie/manager: pass cookiePath to writeChanges()
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service functionality (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
also reordered functions to maintain the hierarchy
2024-10-28 12:08:12 +06:00
wukko 7798844755
api/youtube: refactor, fix fallback, don't repeat same actions
fallback to h264 is now done if there's no required media, not only if adaptive formats list is empty.

best audio and best video are now picked only once.
2024-10-28 12:01:38 +06:00
jj 7dc0121031
api: defer file loads until api is running
Some checks are pending
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
Run service tests / test service: ${{ matrix.service }} (push) Blocked by required conditions
Run service tests / test service functionality (push) Waiting to run
Run tests / check lockfile correctness (push) Waiting to run
Run tests / web sanity check (push) Waiting to run
Run tests / api sanity check (push) Waiting to run
2024-10-27 18:12:59 +00:00