Chomikuj: Download Repacker

Key features:

Despite legitimate uses, Chomikuj is widely known for hosting copyrighted content shared without authorization, similar to RapidShare or Uploaded.net in their prime. chomikuj downloader

This report is for educational and informational purposes only. The author does not endorse unauthorized downloading or copyright infringement. Users should comply with all applicable laws and platform terms. Key features: Despite legitimate uses, Chomikuj is widely

A like JDownloader 2 or the official ChomikBox can be a great way to manage your files. However, stay away from "magic" tools that promise free, unlimited transfers, as they are often fronts for malware. Always prioritize your digital security over a free file. Key features: Despite legitimate uses

Chomikuj: Download Repacker

服务器负载测试工具(st-load)：

1. 模拟huge并发：2G内存就可以开300k连接。基于states-threads的协程。

2. 支持HLS解析和测试，下载ts片后等待一个切片长度，模拟客户端。支持HLS点播和直播。

3. 支持HTTP负载测试，所有并发重复下载一个http文件。可将80Gbps带宽测试的72Gbps。

4. 支持RTMP流测试，一个进程支持5k并发。使用nginx-rtmp的协议直接将chunk流解析为messgae。

state-threads用来模拟超级并发，并简化异步socket的逻辑为同步socket，http-parser解析http协议部分。

这两个库设计都很巧妙，所以我开了一个项目：https://github.com/winlinvip/st-load

state-threads之前就有写过文章说明，那时候主要是支持高并发的rtmp服务器，也是并发和异步变为同步的协程很方便。

http-parser用yum就可以search到，它其实设计得也相当巧妙，相当于只是解析buffer的http内容，并不负责网络部分。libcurl/poco等都带了网络处理，所以不合适。

举例说明，http_parser_parse_url这个函数，解析url，设计得非常有意思，不是返回字符串，而是返回位置索引，譬如主机头在什么位置长度多长等等。

[root@localhost ~]# yum install git unzip patch gcc gcc-c++ make
[root@localhost ~]# git clone https://github.com/winlinvip/st-load.git

[root@localhost st-load]# ./configure
[root@localhost st-load]# make

[root@localhost st-load]# ls objs/
http-parser-2.1 src st_hls_load st_rtmp_load st_rtmp_publish
Makefile st-1.9 st_http_load st_rtmp_load_fast
[root@localhost st-load]#
模拟RTMP用户
./st_rtmp_load -c 1 -r rtmp://127.0.0.1:1935/live/livestream
模拟HLS直播用户
./st_hls_load -c 1 -r http://127.0.0.1:3080/hls/hls.m3u8
模拟HSL点播用户
./st_hls_load -c 10000 -o -r http://127.0.0.1:3080/hls/hls.m3u8
模拟RTMP推流用户
./st_rtmp_publish -i doc/source.200kbps.768×320.flv -c 1 -r rtmp://127.0.0.1:1935/live/livestream
模拟RTMP多路推流用户
./st_rtmp_publish -i doc/source.200kbps.768×320.flv -c 1000 -r rtmp://127.0.0.1:1935/live/livestream_{i}

支持RTMP流播放测试，一个进程支持5k并发
支持RTMP流推流测试，一个进程支持500个并发。

Key features:

Despite legitimate uses, Chomikuj is widely known for hosting copyrighted content shared without authorization, similar to RapidShare or Uploaded.net in their prime.

Chomikuj: Download Repacker

第一种方案：ffmpeg+nginx

新的ffmpeg已经支持HLS。（本人也参与了代码供献，给自己做个广告:)）

点播：

生成hls分片:

ffmpeg -i <媒体文件> -c:v libx264 -c:a -f hls /usr/local/nginx/html/test.m3u8

直播：

ffmpeg -i udp://@:1234 -c:v libx264 -c:a -f hls /usr/local/nginx/html/test.m3u8

建立web服务器：

默认配置就可以。

server {

listen 80;

server_name localhost;

#charset koi8-r;

#access_log logs/host.access.log main;

location / {

root html;

index index.html index.htm;

}

启动nginx。

客户端访问：http://IP/test.m3u8

在windows上可以用vlc播放。

第二个文案，用nginx-rtmp-module

rtmp {

server {

listen 1935;

chunk_size 4000;

#HLS

# For HLS to work please create a directory in tmpfs (/tmp/app here)

# for the fragments. The directory contents is served via HTTP (see

# http{} section in config)

# Incoming stream must be in H264/AAC. For iPhones use baseline H264

# profile (see ffmpeg example).

# This example creates RTMP stream from movie ready for HLS:

# ffmpeg -loglevel verbose -re -i movie.avi –vcodec libx264

# -vprofile baseline -acodec libmp3lame -ar 44100 -ac 1

# -f flv rtmp://localhost:1935/hls/movie

# If you need to transcode live stream use ‘exec’ feature.

application hls {

live on;

hls on;

hls_path /tmp/app;

hls_fragment 5s;

}

http {

server {

listen 80;

location /hls {

# Serve HLS fragments

types {

application/vnd.apple.mpegurl m3u8;

video/mp2t ts;

}

alias /tmp/app;

expires -1;

}

Chomikuj: Download Repacker

相对于 Apache，Nginx 占用的系统资源更少，更适合 VPS 使用。恶意盗链的 User Agent 无处不在，博客更换到 WordPress 没几天，就被 SPAM（垃圾留言）盯上，又被暴力破解后台用户名密码。以前介绍过 Apache 使用 .htaccess 屏蔽恶意 User Agent，今天来介绍 Nginx 屏蔽恶意 User Agent请求的方法。

先上规则&注释

#禁用未初始化变量警告
uninitialized_variable_warn off;
#匹配各种 bad user agent，返回403错误
if ($http_user_agent ~* "embeddedwb|NSPlayer|WMFSDK|qunarbot|mj12bot|ahrefsbot|Windows 98|MSIE 6.0; Windows 2000|EasouSpider|Sogou web spider") {
return 403;
}
#匹配POST方法，给变量iftemp赋值
if ($request_method ~* "POST") {set $iftemp X;}
#匹配 bad user agent，给变量iftemp赋值;这几个UA主要是发垃圾留言的
if ($http_user_agent ~* "MSIE 6.*NET|MSIE 7.*NET|MSIE 6.*SV1|MSIE 6.0; Windows NT 5.0") {
set $iftemp "${iftemp}Y";
}
#如果变量iftemp符合上面两个条件，返回403错误
if ($iftemp = XY) {return 403;}

禁用未初始化变量警告，不然会不停写入警告到错误日志error.log，如下

2014/09/11 09:21:11 [warn] 18649#0: *132 using uninitialized “iftemp” variable, client: 220.181.51.209, server: www.wilf.cn, request: “GET /wp-content/themes/dazzling/inc/fonts/glyphicons-halflings-regular.woff HTTP/1.0”, host: “www.wilf.cn”, referrer: “http://www.wilf.cn/”

2014/09/11 09:21:11 [warn] 18649#0: *92 using uninitialized “iftemp” variable, client: 66.249.79.55, server: www.wilf.cn, request: “GET /page/14?mod=pad&act=view&id=741 HTTP/1.1”, host: “www.wilf.cn”

Nginx 规则不支持2个以上的条件判断，绕个路，通过给变量两次赋值来完成2个条件判断。

Nginx 规则也是使用正则表达式匹配字符串，分析日志，根据需要自己定制。

检验成果的时候到了

183.60.214.51 — [10/Sep/2014:22:16:18 +0800] — Bytes: 13507 — GET /?mod=pad&act=view&id=460 HTTP/1.1 — 403 — – — Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html) — – — –

220.181.125.169 — [11/Sep/2014:09:38:15 +0800] — Bytes: 169 — GET /page/51?mod=wap&act=AddCom&inpId=860 HTTP/1.1 — 403 — – — Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) — – — –

EasouSpider 和 Sogou web spider，再也不见。

http://www.wilf.cn/post/block-bad-user-agent-on-nginx-sever.html