Internal Server Error on Search

Seafile Server
,
1

Hi there,

Having an error on Seafile Pro, on which I receive ‘Internal Server Error’ on the search page.

seahub.log shows the following:

> 020-01-19 09:23:50,549 [WARNING] elasticsearch:97 log_request_fail HEAD http://127.0.0.1:9200/repofiles [status:N/A request:0.001s]
> Traceback (most recent call last):
>   File "/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
>     response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 638, in urlopen
>     _stacktrace=sys.exc_info()[2])
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/util/retry.py", line 343, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 600, in urlopen
>     chunked=chunked)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 354, in _make_request
>     conn.request(method, url, **httplib_request_kw)
>   File "/usr/lib/python2.7/httplib.py", line 1069, in request
>     self._send_request(method, url, body, headers)
>   File "/usr/lib/python2.7/httplib.py", line 1109, in _send_request
>     self.endheaders(body)
>   File "/usr/lib/python2.7/httplib.py", line 1065, in endheaders
>     self._send_output(message_body)
>   File "/usr/lib/python2.7/httplib.py", line 892, in _send_output
>     self.send(msg)
>   File "/usr/lib/python2.7/httplib.py", line 854, in send
>     self.connect()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 181, in connect
>     conn = self._new_conn()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 168, in _new_conn
>     self, "Failed to establish a new connection: %s" % e)
> NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff218e2d550>: Failed to establish a new connection: [Errno 111] Connection refused
> 2020-01-19 09:23:50,575 [WARNING] elasticsearch:97 log_request_fail HEAD http://127.0.0.1:9200/repofiles [status:N/A request:0.001s]
> Traceback (most recent call last):
>   File "/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
>     response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 638, in urlopen
>     _stacktrace=sys.exc_info()[2])
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/util/retry.py", line 343, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 600, in urlopen
>     chunked=chunked)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 354, in _make_request
>     conn.request(method, url, **httplib_request_kw)
>   File "/usr/lib/python2.7/httplib.py", line 1069, in request
>     self._send_request(method, url, body, headers)
>   File "/usr/lib/python2.7/httplib.py", line 1109, in _send_request
>     self.endheaders(body)
>   File "/usr/lib/python2.7/httplib.py", line 1065, in endheaders
>     self._send_output(message_body)
>   File "/usr/lib/python2.7/httplib.py", line 892, in _send_output
>     self.send(msg)
>   File "/usr/lib/python2.7/httplib.py", line 854, in send
>     self.connect()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 181, in connect
>     conn = self._new_conn()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 168, in _new_conn
>     self, "Failed to establish a new connection: %s" % e)
> NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff218e2d650>: Failed to establish a new connection: [Errno 111] Connection refused
> 2020-01-19 09:23:50,576 [WARNING] elasticsearch:97 log_request_fail HEAD http://127.0.0.1:9200/repofiles [status:N/A request:0.001s]
> Traceback (most recent call last):
>   File "/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
>     response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 638, in urlopen
>     _stacktrace=sys.exc_info()[2])
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/util/retry.py", line 343, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 600, in urlopen
>     chunked=chunked)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 354, in _make_request
>     conn.request(method, url, **httplib_request_kw)
>   File "/usr/lib/python2.7/httplib.py", line 1069, in request
>     self._send_request(method, url, body, headers)
>   File "/usr/lib/python2.7/httplib.py", line 1109, in _send_request
>     self.endheaders(body)
>   File "/usr/lib/python2.7/httplib.py", line 1065, in endheaders
>     self._send_output(message_body)
>   File "/usr/lib/python2.7/httplib.py", line 892, in _send_output
>     self.send(msg)
>   File "/usr/lib/python2.7/httplib.py", line 854, in send
>     self.connect()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 181, in connect
>     conn = self._new_conn()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 168, in _new_conn
>     self, "Failed to establish a new connection: %s" % e)
> NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff218e2db90>: Failed to establish a new connection: [Errno 111] Connection refused
> 2020-01-19 09:23:50,577 [WARNING] elasticsearch:97 log_request_fail HEAD http://127.0.0.1:9200/repofiles [status:N/A request:0.001s]
> Traceback (most recent call last):
>   File "/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
>     response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 638, in urlopen
>     _stacktrace=sys.exc_info()[2])
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/util/retry.py", line 343, in increment
>     raise six.reraise(type(error), error, _stacktrace)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 600, in urlopen
>     chunked=chunked)
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connectionpool.py", line 354, in _make_request
>     conn.request(method, url, **httplib_request_kw)
>   File "/usr/lib/python2.7/httplib.py", line 1069, in request
>     self._send_request(method, url, body, headers)
>   File "/usr/lib/python2.7/httplib.py", line 1109, in _send_request
>     self.endheaders(body)
>   File "/usr/lib/python2.7/httplib.py", line 1065, in endheaders
>     self._send_output(message_body)
>   File "/usr/lib/python2.7/httplib.py", line 892, in _send_output
>     self.send(msg)
>   File "/usr/lib/python2.7/httplib.py", line 854, in send
>     self.connect()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 181, in connect
>     conn = self._new_conn()
>   File "/data/seafile/seafile-pro-server-7.0.9/seahub/thirdpart/urllib3/connection.py", line 168, in _new_conn
>     self, "Failed to establish a new connection: %s" % e)
> NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff218e2d550>: Failed to establish a new connection: [Errno 111] Connection refused
> 2020-01-19 09:23:50,578 [ERROR] seahub.api2.views:576 get ConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff218e2d550>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff218e2d550>: Failed to establish a new connection: [Errno 111] Connection refused)

I saw some other posts that made reference to elasticsearch logs, however nothing has come up in those. The last entry in the elasticsearch.log is from 5 days ago which had a ‘high disk watermark exceeded’ error – my disk was full a couple of days ago, but i have expanded the drive size now, and there is still 80GB available.

Any clues?

2

The search uses elasticsearch. It looks like elasticsearch is either not running at all or at least not accepting connections (Failed to establish a new connection).

3

Ok thanks. Based on some web searching I did a test from the command line to see if elasticsearch was running:

root@x:/data/seafile# curl -X GET http://localhost:9200
curl: (7) Failed to connect to localhost port 9200: Connection refused

now doing a

netstat -a | grep 9200

returns no results.

i’m a bit stuck as to how to get elasticsearch to start running, as my searches so far point to restarting the ‘elasticsearch’ service (using ubuntu FYI), however elasticsearch is an unrecognized service:

root@x:/data/seafile# service elasticsearch status
Unit elasticsearch.service could not be found.

would anyone have anything further to point me in the right direction?

thanks.

4

Seafile delivers a bundled version of elasticsearch. It should be within the install directory. I’m not into how it is started an managed, though.

5

There is definitely elasticsearch folders in the install directory:

/data/seafile/seafile-pro-server-7.0.9/pro/elasticsearch/
/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch/
/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch-5.5.0.dist-info/
/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch_dsl/
/data/seafile/seafile-pro-server-7.0.9/pro/python/elasticsearch_dsl-5.4.0.dist-info/

i’m hoping one of the devs could maybe offer some insight? otherwise i might have to log an email directly with them as i’m out of ideas at this point.

6

Hi adam,

I ran into the same issue before. The cause of this for me is quite simple.

First, check if elasticsearch process is running:

~$ ps -aux | grep elastic

You shouldn’t see any process related with elasticsearch.
Then, check if the pid exists.

~$ ls haiwen/pids/elasticsearch.pid

The pid should exist.

If the above two steps match what I describe, then congrats, you ran into the same issue as I have.

Solution is pretty straightforward: delete the haiwen/pids/elasticsearch.pid, and restart seafile server. Check the elasticsearch process again and it should be running. Then seafile should get back to normal.

3 Likes
7

Fantastic, thanks for your help.

Deleting the elasticsearch.pid has fixed the issue for me.

Much appreciated!

8

This issue could be related to Elasticsearch not running or not being accessible.

Check if the Elasticsearch service is running on your server. You can use the following command to check the status:
systemctl status elasticsearch
Restart Elasticsearch: If Elasticsearch is running, try restarting the service to ensure it’s in a good state
systemctl restart elasticsearch
Check the Elasticsearch logs for any errors or warnings that could provide more information about the issue. The logs are typically located in the /var/log/elasticsearch/ directory.

Double-check the Elasticsearch connection settings in your Seafile configuration files (seahub_settings.py or ccnet.conf ). Ensure that the Elasticsearch host and port specified in the configuration match the actual Elasticsearch server.