627 lines
28 KiB
HTML
627 lines
28 KiB
HTML
<!DOCTYPE html>
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-us" lang="en-us">
|
|
<head>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<script defer language="javascript" type="text/javascript" src="/js/bundle.min.14549c76bbc96f0af1574b0259efd70e52908cd36fb4d14ed3d290a1b6479eae.js"></script>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<meta http-equiv="content-type" content="text/html; charset=utf-8">
|
|
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
|
|
|
|
<link rel="icon" href=images/tiger.png>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<!-- Open Graph image and Twitter Card metadata -->
|
|
|
|
<title itemprop="name">tinfoil-hat.net - Nitter Ratelimiting & Guest_Accounts branch</title>
|
|
<meta property="og:title" content=tinfoil-hat.net - Nitter Ratelimiting & Guest_Accounts branch />
|
|
<meta name="twitter:title" content=tinfoil-hat.net - Nitter Ratelimiting & Guest_Accounts branch />
|
|
<meta itemprop="name" content=tinfoil-hat.net - Nitter Ratelimiting & Guest_Accounts branch />
|
|
<meta name="application-name" content=tinfoil-hat.net - Nitter Ratelimiting & Guest_Accounts branch />
|
|
<meta property="og:site_name" content="" />
|
|
|
|
|
|
<meta name="description" content="" />
|
|
<meta itemprop="description" content="" />
|
|
<meta property="og:description" content="" />
|
|
<meta name="twitter:description" content="" />
|
|
|
|
|
|
<base href="/posts/nitter-ratelimiting/" />
|
|
<link rel="canonical" href="/posts/nitter-ratelimiting/" itemprop="url" />
|
|
<meta name="url" content="/posts/nitter-ratelimiting/" />
|
|
<meta name="twitter:url" content="/posts/nitter-ratelimiting/" />
|
|
<meta property="og:url" content="/posts/nitter-ratelimiting/" />
|
|
|
|
|
|
<meta property="og:updated_time" content="2023-10-08T00:01:08+02:00" />
|
|
|
|
|
|
<link rel="sitemap" type="application/xml" title="Sitemap" href='/sitemap.xml' />
|
|
|
|
<meta name="robots" content="index,follow" />
|
|
<meta name="googlebot" content="index,follow" />
|
|
|
|
|
|
<meta name="twitter:site" content="" />
|
|
<meta name="twitter:creator" content="" />
|
|
<meta property="fb:admins" content="" />
|
|
|
|
|
|
<meta name="apple-mobile-web-app-title" content="" />
|
|
<meta name="apple-mobile-web-app-capable" content="yes" />
|
|
<meta name="apple-mobile-web-app-status-bar-style" content="black" />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<meta name="generator" content="Hugo 0.111.3">
|
|
|
|
|
|
|
|
|
|
|
|
<link type="text/css" rel="stylesheet" href="/css/bundle.min.94a339836f89f0d25f31980cb6b0631da21e20af128308747ce44e0525eb16ef.css">
|
|
|
|
|
|
|
|
<style>
|
|
body {
|
|
--sidebar-bg-color: #202020;
|
|
--sidebar-img-border-color: #515151;
|
|
--sidebar-p-color: #909090;
|
|
--sidebar-h1-color: #FFF;
|
|
--sidebar-a-color: #FFF;
|
|
--sidebar-socials-color: #FFF;
|
|
--text-color: #222;
|
|
--bkg-color: #FAF9F6;
|
|
--post-title-color: #303030;
|
|
--list-color: #5A5A5A;
|
|
--link-color: #268BD2;
|
|
--date-color: #515151;
|
|
--table-border-color: #E5E5E5;
|
|
--table-stripe-color: #F9F9F9;
|
|
--code-color: #000;
|
|
--code-background-color: #E5E5E5;
|
|
--code-block-color: #FFF;
|
|
--code-block-background-color: #272822;
|
|
--moon-sun-color: #FFF;
|
|
--moon-sun-background-color: #515151;
|
|
}
|
|
body.dark-theme {
|
|
--text-color: #EEE;
|
|
--bkg-color: #121212;
|
|
--post-title-color: #DBE2E9;
|
|
--list-color: #9D9D9D;
|
|
--link-color: #268BD2;
|
|
--date-color: #9A9A9A;
|
|
--table-border-color: #515151;
|
|
--table-stripe-color: #202020;
|
|
--code-color: #FFF;
|
|
--code-background-color: #515151;
|
|
--code-block-color: #FFF;
|
|
--code-block-background-color: #272822;
|
|
}
|
|
body {
|
|
background-color: var(--bkg-color);
|
|
}
|
|
</style>
|
|
|
|
</head>
|
|
|
|
<body class="">
|
|
<div class="wrapper">
|
|
<aside class="sidebar">
|
|
<div class="container sidebar-sticky">
|
|
<div class="light-dark" align="right">
|
|
<button class="btn-light-dark" title="Toggle light/dark mode">
|
|
<svg class="moon" xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 16 16">
|
|
<path fill="currentColor" d="M6 .278a.768.768 0 0 1 .08.858a7.208 7.208 0 0 0-.878 3.46c0 4.021 3.278 7.277 7.318 7.277c.527 0 1.04-.055 1.533-.16a.787.787 0 0 1 .81.316a.733.733 0 0 1-.031.893A8.349 8.349 0 0 1 8.344 16C3.734 16 0 12.286 0 7.71C0 4.266 2.114 1.312 5.124.06A.752.752 0 0 1 6 .278z"/>
|
|
</svg>
|
|
<svg class="sun" xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 16 16">
|
|
<path fill="currentColor" d="M8 12a4 4 0 1 0 0-8a4 4 0 0 0 0 8zM8 0a.5.5 0 0 1 .5.5v2a.5.5 0 0 1-1 0v-2A.5.5 0 0 1 8 0zm0 13a.5.5 0 0 1 .5.5v2a.5.5 0 0 1-1 0v-2A.5.5 0 0 1 8 13zm8-5a.5.5 0 0 1-.5.5h-2a.5.5 0 0 1 0-1h2a.5.5 0 0 1 .5.5zM3 8a.5.5 0 0 1-.5.5h-2a.5.5 0 0 1 0-1h2A.5.5 0 0 1 3 8zm10.657-5.657a.5.5 0 0 1 0 .707l-1.414 1.415a.5.5 0 1 1-.707-.708l1.414-1.414a.5.5 0 0 1 .707 0zm-9.193 9.193a.5.5 0 0 1 0 .707L3.05 13.657a.5.5 0 0 1-.707-.707l1.414-1.414a.5.5 0 0 1 .707 0zm9.193 2.121a.5.5 0 0 1-.707 0l-1.414-1.414a.5.5 0 0 1 .707-.707l1.414 1.414a.5.5 0 0 1 0 .707zM4.464 4.465a.5.5 0 0 1-.707 0L2.343 3.05a.5.5 0 1 1 .707-.707l1.414 1.414a.5.5 0 0 1 0 .708z"/>
|
|
</svg>
|
|
</button>
|
|
</div>
|
|
|
|
<div class="sidebar-about">
|
|
<h1 class="brand">
|
|
|
|
<a href="/">
|
|
<img src="/images/tiger.png" alt="brand image">
|
|
</a>
|
|
|
|
|
|
<a href="/">
|
|
<h1>tinfoil-hat.net</h1>
|
|
</a>
|
|
|
|
</h1>
|
|
<p class="lead">
|
|
A secure home for every tinfoil-hat
|
|
</p>
|
|
</div>
|
|
|
|
<nav>
|
|
<ul class="sidebar-nav">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="heading">
|
|
<a href="/posts/">Posts</a>
|
|
</li>
|
|
|
|
<li class="sub-heading">
|
|
Recent
|
|
</li>
|
|
|
|
<li class="bullet">
|
|
<a href="/posts/gpu-passtrough-mint21/">Gpu Passtrough Mint21</a>
|
|
</li>
|
|
|
|
<li class="bullet">
|
|
<a href="/posts/vbox-autostart/">Vbox Autostart</a>
|
|
</li>
|
|
|
|
<li class="bullet">
|
|
<a href="/posts/bst/">BST</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="heading">
|
|
<a href="/unixporn/">Unixporn</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="heading">
|
|
<a href="/webring/">Webring</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
</nav>
|
|
|
|
|
|
|
|
<a target="_blank" class="social" title="Gitlab" href="https://git.tinfoil-hat.net">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="1.2em" height="1.2em" viewBox="-2 -2.5 24 24">
|
|
<path fill="currentColor" d='M10.006 18.443L6.326 7.118h7.36l-3.68 11.325zm0 0L1.168 7.118h5.158l3.68 11.325zM1.168 7.118l8.838 11.325-9.68-7.032a.762.762 0 0 1-.276-.852l1.118-3.441zm0 0L3.385.296a.38.38 0 0 1 .724 0l2.217 6.822H1.168zm8.838 11.325l3.68-11.325h5.157l-8.837 11.325zm8.837-11.325l1.119 3.441a.762.762 0 0 1-.277.852l-9.68 7.032 8.838-11.325zm0 0h-5.157L15.902.296a.38.38 0 0 1 .725 0l2.216 6.822z' />
|
|
|
|
</svg>
|
|
</a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<a target="_blank" class="social" title="YouTube" href="https://tube.tinfoil-hat.net">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="1.2em" height="1.2em" viewBox="0 0 24 24">
|
|
<path fill="currentColor" d="M12.006 19.012h-.02c-.062 0-6.265-.012-7.83-.437a2.5 2.5 0 0 1-1.764-1.765A26.494 26.494 0 0 1 1.986 12a26.646 26.646 0 0 1 .417-4.817A2.564 2.564 0 0 1 4.169 5.4c1.522-.4 7.554-.4 7.81-.4H12c.063 0 6.282.012 7.831.437c.859.233 1.53.904 1.762 1.763c.29 1.594.427 3.211.407 4.831a26.568 26.568 0 0 1-.418 4.811a2.51 2.51 0 0 1-1.767 1.763c-1.52.403-7.553.407-7.809.407Zm-2-10.007l-.005 6l5.212-3l-5.207-3Z"/>
|
|
</svg>
|
|
</a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<a target="_blank" class="social" title="RSS Feed" href="/posts/index.xml">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1.2em" viewBox="0 0 1280.000000 1280.000000">
|
|
<g transform="translate(0.000000,1280.000000) scale(0.100000,-0.100000)" fill="currentColor">
|
|
<path d="M2295 11929 c-284 -12 -642 -45 -707 -65 -17 -5 -18 -63 -18 -1039 0 -569 4 -1036 8 -1039 5 -3 74 6 153 19 510 86 1168 95 1789 25 1348 -153 2602 -677 3670 -1531 385 -308 820 -744 1126 -1129 842 -1060 1362 -2313 1514 -3650 70 -621 61 -1279 -25 -1789 -13 -79 -22 -148 -19 -153 3 -4 471 -8 1039 -8 l1035 0 5 23 c51 225 85 942 67 1419 -23 605 -77 1044 -198 1617 -294 1400 -927 2734 -1823 3846 -1043 1295 -2364 2259 -3909 2854 -1158 447 -2451 656 -3707 600z"/>
|
|
<path d="M2255 7845 c-269 -25 -620 -81 -667 -106 -17 -9 -18 -55 -18 -899 0 -706 3 -890 13 -890 6 0 66 18 132 41 130 44 288 79 467 105 154 21 577 30 749 15 1207 -107 2267 -823 2814 -1902 166 -327 268 -637 330 -1001 38 -227 48 -384 42 -662 -8 -348 -44 -590 -126 -831 -23 -66 -41 -126 -41 -132 0 -10 184 -13 890 -13 844 0 890 1 899 18 27 50 88 452 110 725 14 162 14 624 1 782 -59 703 -233 1323 -545 1945 -481 956 -1313 1788 -2270 2268 -620 310 -1239 483 -1940 542 -165 14 -669 10 -840 -5z"/>
|
|
<path d="M2519 3815 c-391 -66 -725 -336 -868 -703 -79 -201 -96 -462 -45 -677 83 -344 338 -641 666 -774 116 -47 205 -69 330 -80 412 -39 811 153 1040 500 193 292 240 648 128 981 -135 403 -492 699 -914 757 -100 14 -241 12 -337 -4z"/>
|
|
</g>
|
|
</svg>
|
|
</a>
|
|
|
|
|
|
<a target="_blank" class="social" title="Email" href="mailto://mail@tinfoil-hat.net">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1.2em" viewBox="0 0 485.211 485.211">
|
|
<path fill="currentColor" d="M301.393,241.631L464.866,424.56H20.332l163.474-182.928l58.801,51.443L301.393,241.631z M462.174,60.651H23.027 l219.579,192.142L462.174,60.651z M324.225,221.67l160.986,180.151V80.792L324.225,221.67z M0,80.792v321.029L160.972,221.64 L0,80.792z"/>
|
|
</svg>
|
|
</a>
|
|
|
|
|
|
<p class="footnote">
|
|
powered by <a target="_blank" href="https://gohugo.io">Hugo</a> | themed with <a target="_blank" href="https://github.com/lukeorth/poison">poison</a>
|
|
<br>
|
|
© 2024 . All rights reserved.
|
|
</p>
|
|
|
|
</div>
|
|
</aside>
|
|
|
|
<main class="content container">
|
|
<div class="post">
|
|
<div class="info">
|
|
<h1 class="post-title">
|
|
<a href="/posts/nitter-ratelimiting/">Nitter Ratelimiting & Guest_Accounts branch</a>
|
|
</h1>
|
|
|
|
<div class="headline">
|
|
<div>
|
|
|
|
<time datetime=" 2023-10-08T00:01:08+0200" class="post-date">
|
|
October 8, 2023
|
|
</time>
|
|
|
|
<span> - </span>
|
|
<span class="reading-time">
|
|
|
|
|
|
|
|
|
|
<span>3 mins read</span>
|
|
</span>
|
|
</div>
|
|
|
|
|
|
<ul class="tags">
|
|
|
|
<li class="tag-Technology">
|
|
<a href="/tags/technology">Technology</a>
|
|
</li>
|
|
|
|
<li class="tag-Linux">
|
|
<a href="/tags/linux">Linux</a>
|
|
</li>
|
|
|
|
<li class="tag-Server">
|
|
<a href="/tags/server">Server</a>
|
|
</li>
|
|
|
|
</ul>
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
</div>
|
|
|
|
<h1 id="mitigating-unauthorized-web-scraping-bot-traffic">Mitigating Unauthorized Web Scraping Bot Traffic</h1>
|
|
<p>The current design of Nitter, along with its methodology for accessing the Twitter service, necessitates heightened vigilance on the part of instance operators to manage unwarranted access by web scraping bots. This guide is intended to provide operators with essential information to effectively mitigate unauthorized web scraping bot traffic.</p>
|
|
<h2 id="prerequisites">Prerequisites</h2>
|
|
<p>Before proceeding with the rate-limiting setup, ensure that you have:</p>
|
|
<ol>
|
|
<li>A functional Nitter installation located at <code>/opt/nitter</code>.</li>
|
|
<li>Nginx as your web server, with the server block as outlined in <a href="https://github.com/zedeus/nitter/wiki/Nginx" target="_blank">Nginx Configuration</a>. In this guide, we refer to this server block as ’nitter.nginx.'</li>
|
|
</ol>
|
|
<h2 id="rate-limiting-configuration">Rate Limiting Configuration</h2>
|
|
<p>Navigate to <code>/etc/nginx</code> and create two necessary files with the following content:</p>
|
|
<p><strong>shared_cache.conf</strong></p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-nginx" data-lang="nginx"><span style="display:flex;"><span><span style="color:#66d9ef">proxy_buffers</span> <span style="color:#ae81ff">64</span> <span style="color:#ae81ff">16k</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">proxy_buffer_size</span> <span style="color:#ae81ff">4k</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">expires</span> <span style="color:#e6db74">90d</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">access_log</span> <span style="color:#66d9ef">off</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">proxy_pass</span> <span style="color:#e6db74">http://127.0.0.1:8080</span>;
|
|
</span></span></code></pre></div><p><strong>shared_static.conf</strong></p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-nginx" data-lang="nginx"><span style="display:flex;"><span><span style="color:#66d9ef">expires</span> <span style="color:#e6db74">90d</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">access_log</span> <span style="color:#66d9ef">off</span>;
|
|
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">root</span> <span style="color:#e6db74">/opt/nitter/public</span>
|
|
</span></span></code></pre></div><p>These files ensure that normal usage, such as serving images, videos, and site data, does not trigger rate limiting. Logging for these locations is disabled to prevent entries in access or error logs.</p>
|
|
<p>Next, add the rate-limiting rules within the nginx.conf file, located within the http block:</p>
|
|
<p><strong>nginx.conf</strong></p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-nginx" data-lang="nginx"><span style="display:flex;"><span><span style="color:#66d9ef">http</span> {
|
|
</span></span><span style="display:flex;"><span>
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">limit_req_zone</span> $binary_remote_addr <span style="color:#e6db74">zone=nitter.tld_sec:10m</span> <span style="color:#e6db74">rate=1r/s</span>;
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">limit_req_zone</span> $binary_remote_addr <span style="color:#e6db74">zone=nitter.tld_min:10m</span> <span style="color:#e6db74">rate=45r/m</span>;
|
|
</span></span><span style="display:flex;"><span>}
|
|
</span></span></code></pre></div><p>These settings limit users to one request per second and 45 requests per minute, a natural browsing rate for the site.</p>
|
|
<p>Now, in your ’nitter.nginx’ server block:</p>
|
|
<p><strong>nitter.nginx</strong></p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-nginx" data-lang="nginx"><span style="display:flex;"><span><span style="color:#66d9ef">server</span> {
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> <span style="color:#e6db74">/pic/</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_cache.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> <span style="color:#e6db74">/video/</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_cache.conf</span>; }
|
|
</span></span><span style="display:flex;"><span>
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> <span style="color:#e6db74">/css/</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> <span style="color:#e6db74">/js/</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> <span style="color:#e6db74">/fonts/</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/apple-touch-icon.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/apple-touch-icon-precomposed.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/android-chrome-192x192.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/favicon-32x32.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/favicon-16x16.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/favicon.ico</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/logo.png</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">location</span> = <span style="color:#e6db74">/site.webmanifest</span> { <span style="color:#f92672">include</span> <span style="color:#e6db74">shared_static.conf</span>; }
|
|
</span></span><span style="display:flex;"><span>}
|
|
</span></span></code></pre></div><p><strong>nitter.nginx</strong></p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-nginx" data-lang="nginx"><span style="display:flex;"><span> <span style="color:#66d9ef">location</span> <span style="color:#e6db74">/</span> {
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">proxy_pass</span> <span style="color:#e6db74">http://localhost:8080</span>;
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">limit_req</span> <span style="color:#e6db74">zone=nitter.tld_sec</span> <span style="color:#e6db74">burst=3</span> <span style="color:#e6db74">nodelay</span>;
|
|
</span></span><span style="display:flex;"><span> <span style="color:#f92672">limit_req</span> <span style="color:#e6db74">zone=nitter.tld_min</span> <span style="color:#e6db74">burst=4</span>;
|
|
</span></span><span style="display:flex;"><span> }
|
|
</span></span></code></pre></div><p>The ‘burst’ parameter allows for temporary bursts of traffic while maintaining the overall rate limit, ensuring a smoother user experience while preventing server overload and misuse.</p>
|
|
<p>Reload Nginx configuration files:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>nginx -s reload
|
|
</span></span></code></pre></div><p>Nginx will now rate limit IP for excessive usage.</p>
|
|
<h3 id="fail2ban"><strong>Fail2ban</strong></h3>
|
|
<p>To implement rate limiting and address repeat offenders, a functional install of Fail2ban (<a href="https://github.com/fail2ban/fail2ban" target="_blank">https://github.com/fail2ban/fail2ban</a>) is required. Typically, Fail2ban configuration files are located at /etc/fail2ban. Make a copy of jail.conf named jail.local, as Fail2ban will prioritize jail.local by default when both are present:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
|
|
</span></span></code></pre></div><p>Within jail.local, ensure that bantime.increment is uncommented and set to true:</p>
|
|
<pre tabindex="0"><code class="language-fail2ban" data-lang="fail2ban">bantime.increment = true
|
|
</code></pre><p>Additionally, enable the rate-limiting block in jail.local by setting ’enabled’ to true:</p>
|
|
<pre tabindex="0"><code class="language-fail2ban" data-lang="fail2ban">[nginx-limit-req]
|
|
enabled = true
|
|
port = http,https
|
|
logpath = %(nginx_error_log)s
|
|
</code></pre><p>Restart Fail2ban:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>systemctl restart fail2ban
|
|
</span></span></code></pre></div><p>To verify that the jail is running:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>fail2ban-client status nginx-limit-req
|
|
</span></span></code></pre></div><p>You’ll receive a summary of failures triggered by the filter and the number of active actions.</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>Status <span style="color:#66d9ef">for</span> the jail: nginx-limit-req
|
|
</span></span><span style="display:flex;"><span>|- Filter
|
|
</span></span><span style="display:flex;"><span>| |- Currently failed: <span style="color:#ae81ff">0</span>
|
|
</span></span><span style="display:flex;"><span>| |- Total failed: <span style="color:#ae81ff">0</span>
|
|
</span></span><span style="display:flex;"><span>| <span style="color:#e6db74">`</span>- File list: %<span style="color:#f92672">(</span>nginx_error_log<span style="color:#f92672">)</span>s
|
|
</span></span><span style="display:flex;"><span><span style="color:#e6db74">`</span>- Actions
|
|
</span></span><span style="display:flex;"><span> |- Currently banned: <span style="color:#ae81ff">0</span>
|
|
</span></span><span style="display:flex;"><span> |- Total banned: <span style="color:#ae81ff">0</span>
|
|
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">`</span>- Banned IP list:
|
|
</span></span></code></pre></div><p>Fail2ban will now enforce rate limits, with incremental punishments for repeat infractions when bans expire.</p>
|
|
<hr>
|
|
<hr>
|
|
<h1 id="switch-to-guest_accounts-compile-it-as-normal">Switch to guest_accounts, compile it as normal</h1>
|
|
<pre tabindex="0"><code>git clone https://github.com/zedeus/nitter
|
|
cd nitter
|
|
git branch guest_accounts
|
|
nimble build -d:release
|
|
nimble scss
|
|
nimble md
|
|
</code></pre><pre tabindex="0"><code>touch guest_accounts.jsonl
|
|
vim guest_accounts.jsonl
|
|
</code></pre><p>paste <code>[]</code> inside it</p>
|
|
<h1 id="finally-apply-for-guest-accounts-here">Finally apply for guest accounts here:</h1>
|
|
<p><a href="https://twitterminator.x86-64-unknown-linux-gnu.zip/" target="_blank">https://twitterminator.x86-64-unknown-linux-gnu.zip/</a></p>
|
|
<h2 id="create-cronjob">Create cronjob</h2>
|
|
<pre tabindex="0"><code>0 */4 * * * curl ...guest_accounts.jsonl
|
|
</code></pre>
|
|
|
|
<hr>
|
|
<div class="footer">
|
|
|
|
|
|
<a class="previous-post" href="/posts/till-lindemann-lizard/?ref=footer"><span style="font-weight:bold;">« Previous</span><br>Rammstein Singer Till Lindemann Lizard Confirmed</a>
|
|
|
|
|
|
<div class="next-post">
|
|
<a href="/posts/the-black-and-the-white-wolf/?ref=footer"><span style="font-weight:bold;">Next »</span><br>The Black and the White Wolf</a>
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</div>
|
|
</main>
|
|
|
|
|
|
<div class="article-toc ">
|
|
<div class="toc-wrapper">
|
|
<h4 id="contents"></h4>
|
|
<nav id="TableOfContents">
|
|
<ul>
|
|
<li><a href="#prerequisites">Prerequisites</a></li>
|
|
<li><a href="#rate-limiting-configuration">Rate Limiting Configuration</a>
|
|
<ul>
|
|
<li><a href="#fail2ban"><strong>Fail2ban</strong></a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
|
|
<ul>
|
|
<li><a href="#create-cronjob">Create cronjob</a></li>
|
|
</ul>
|
|
</nav>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
|
|
</div>
|
|
</body>
|
|
</html>
|