[Evergreen-general] Question about search engine bots & DB CPU spikes

[Jason Stephenson]

On 12/2/21 10:07 PM, JonGeorg SageLibrary via Evergreen-general wrote:
> I tried that and still got the loopback address, after restarting 
> services. Any other ideas? And the robots.txt file seems to be doing 
> nothing, which is not much of a surprise. I've reached out to the people 
> who host our network and have control of everything on the other side of 
> the firewall.
> -Jon

I am going to repeat the instructions with more precision to make sure 
that there is no misunderstanding:

In /etc/apache2/eg_vhost.conf, find the following 3 lines and remove the 
leading #:

#RemoteIPInternalProxy 127.0.0.1/24
#RemoteIPInternalProxy ::1
#RemoteIPHeader X-Forwarded-For

In /etc/nginx/sites-enabled/osrf-ws-http-proxy, make sure that the 
following lines appear in BOTH "location / { ... }" blocks, i.e. the are 
between the opening { and closing }:

         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header X-Forwarded-Proto $scheme;

Run the following commands on the command line, my example runs all 3 in 
a single go if you copy and paste the whole thing:

sudo bash <<EOF
a2enmod remoteip
systemctl restart apache2
systemctl restart nginx
EOF

Remote IP addresses should start showing up in the Apache logs for 
regular HTTP and HTTPS requests. It will NOT get you the remote IP 
addresses of clients using websockets.

The above assumes you're running on a recent Debian or Ubuntu with 
systemd installed; that you've followed the Evergreen installation 
instructions, and that you have not already heavily modified the Apache 
or nginx sample configurations.

You should not have to touch the OpenSRF services for this.

Good luck,
Jason