Update 6.advanced-concepts.md

This commit is contained in:
Pas 2025-07-21 15:42:17 -06:00
parent 8ae73e86d7
commit ad8db7b315

View file

@ -16,6 +16,60 @@ Modern streaming services use various protection mechanisms.
### Handling Protected Streams
**Convert HLS playlists to data URLs:**
Data URLs embed content directly in the URL using base64 encoding, completely bypassing CORS restrictions since no HTTP request is made to a different origin. This is often the most effective solution for HLS streams.
```typescript
import { convertPlaylistsToDataUrls } from '@/utils/playlist';
// Original playlist URL from streaming service
const playlistUrl = 'https://protected-cdn.example.com/playlist.m3u8';
// Headers required to access the playlist
const headers = {
'Referer': 'https://player.example.com/',
'Origin': 'https://player.example.com',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
};
// Convert playlist and all variants to data URLs
const dataUrl = await convertPlaylistsToDataUrls(
ctx.proxiedFetcher,
playlistUrl,
headers
);
return {
stream: [{
id: 'primary',
type: 'hls',
playlist: dataUrl, // Self-contained data URL
flags: [flags.CORS_ALLOWED], // No CORS issues with data URLs
captions: []
}]
};
```
**Why data URLs work for CORS bypass:**
- **Inital proxied request WITH HEADERS**: A request is sent from the proxy *with* headers allowing for the playlist to load
- **Fewer external requests**: Content is embedded directly in the URL as base64
- **Same-origin**: Browsers treat data URLs as same-origin content
- **Complete isolation**: No network requests means no CORS preflight checks
- **Self-contained**: All playlist data and segments are embedded in the response
**How the conversion works:**
1. Fetches the master playlist using provided headers
2. For each quality variant, fetches the variant playlist
3. Converts all playlists to base64-encoded data URLs
4. Returns a master data URL containing all embedded variants
**When to use data URLs vs M3U8 proxy:**
- **Use data URLs** when you can fetch all playlist data upfront
- **Use M3U8 proxy** when playlists are too large or change frequently
- **Data URLs are preferred** for most HLS streams due to simplicity and reliability
- **Each segment is origin or header locked**: converting to a data URL does not apply the headers to the segments
**Use M3U8 proxy for HLS streams:**
Using the createM3U8ProxyUrl function we can use our configured M3U8 proxy to send headers to the playlist and all it's segments.
@ -49,7 +103,7 @@ return {
### Stream Validation Bypass
When using M3U8 proxies that are origin-locked (like P-Stream proxies), you may need to bypass automatic stream validation in `valid.ts`:
Streams are checked with a proxied fetcher before returning. However, some streams may be blocked by proxies, so you may need to bypass automatic stream validation in `valid.ts`:
```typescript
// In src/utils/valid.ts, add your scraper ID to skip validation
@ -62,18 +116,17 @@ const SKIP_VALIDATION_CHECK_IDS = [
**Why this is needed:**
- By default, all streams are validated by attempting to fetch metadata
- The validation uses `proxiedFetcher` to check if streams are playable
- If your proxy blocks the fetcher (origin-locked), validation will fail
- If the stream blocks the fetcher, validation will fail
- But the proxied URL should still work in the actual player context
- Adding to skip list bypasses validation and returns the proxied URL directly without checking it
**When to skip validation:**
- Your scraper uses origin-locked proxies
- The proxy service blocks programmatic access
- Validation consistently fails but streams work in browsers
- You're certain the proxy setup is correct
- The stream may be origin or IP locked
- The stream blocks the extension or proxy
**Use setupProxy for MP4 streams:**
When adding headers in the stream response, usually may need to use the extension or native to send the correct headers in the request.
When adding headers in the stream response, usually may need to use the **extension** or native to send the correct headers in the request.
```typescript
import { setupProxy } from '@/utils/proxy';