SECURITY ADVISORY: Seafile Server's vulnerability to "Log4Shell" (log4j vulnerability, CVE-2021-44228)

German version below…

English version (last update at 12/17/2021, 1:30am)

Here is the current status of our analyses on the log4j vulnerability - CVE-2021-44228:

Important: the issue is dynamic and needs to be actively monitored further. Statements that are still current today may no longer be valid tomorrow. However, we will continue to inform you here in any case.

Brief description of the vulnerability:

The vulnerability in log4j exploits the unintended processing of manipulated log entries. Manipulated log entries allow the reloading and execution of malicious code downloaded over the Internet. In the worst case, this vulnerability allows an attacker to gain root privileges on the system and execute code, e.g. to build a backdoor.

Seafile Professional Edition (PE) version 7 and 8 uses elasticsearch 5.6 bundled with one of log4j version 2.11 for full-text search. Seafile PE version 6 uses elasticsearch 2.4.5 with log4j version 1.2. Both versions of log4j are now considered vulnerable. Thus, a Seafile server is basically threatened.

Threat situation for Seafile Server Professional Edition:

According to the current assessment, the threat situation is moderate. The attack cannot be triggered directly by a URL call, but it can be exploited by searching published libraries, i.e. even without a user account. If no libraries are published, an attack can only be carried out by a logged-in user, either through a manipulated file or a query in the full-text search.

Furthermore, you are NOT affected if ONE of the following conditions applies:

  • You are using Seafile Server Community Edition (CE does not use elasticsearch)
  • You have disabled full text search in seafevents.conf ([INDEX FILES] enabled = false)
  • Your Seafile server is not allowed to reload code from the Internet

If none of these conditions apply, the vulnerability can generally be exploited. Log entries in seahub.log indicate that attempts were made to use this vulnerability via HTTP request. Such attacks are not possible as shown above.

Important new finding:
The following two conditions are not sufficient after all and do not provide complete protection:

Starting points to exploit the vulnerability

The following graphic helps to understand the possible mitigations.

Mitigation for Seafile 7.0 and newer:

According to current knowledge, there are only two ways to protect against the log4j vulnerability:

A) the safest option is to disable Seafile’s full-text search. In this case, no Java processes are executed and no malicious code can be loaded. This corresponds to case 2 “DISABLE LOG4J” in the graphic above.

B) the second option is to patch the critical JNDI lookup command. For this it is sufficient to execute the following command in the directory /seafile-server-latest/pro/elasticsearch/lib/:

zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

This corresponds to point 3 in the graphic “DISABLE JNDI LOOKUPS”.

(Source: https://blog.cloudflare.com/inside-the-log4j2-vulnerability-cve-2021-44228/)

This command has already been tested by us on several Seafile 7.x and 8.x versions without limiting the functionality of Seafile or the full-text search.

Mitigation for Seafile Server v6.3 and older

We recommend updating to at least Seafile Server Professional Edition 7.1 or newer and the recommended approach for this version.

Scan for vulnerable log4j classes

LunaSec provide a tool on Github to scan for vulnerable log4j classes (https://github.com/lunasec-io/lunasec/releases/). You can run the scan yourself on your Seafile server with the following commands:

mkdir /tmp/log4jscan
cd /tmp/log4jscan
wget https://github.com/lunasec-io/lunasec/releases/download/v1.3.1-log4shell/log4shell_1.3.1-log4shell_Linux_x86_64
chmod +x log4sheel_1.3.1-log4shell_Linux_x86_64
./log4sheel_1.3.1-log4shell_Linux_x86_64 scan /home/seafile/seafile-server-latest/

Notes:

  • these commands are for a 64-bit Linux server. Adjust the link in the wget command accordingly if you have a different system
  • if your seafile is not installed in /home/seafile/, adjust the path accordingly
  • do not forget the “/” at the end of the path. Otherwise the scan command will not work.

The result will look like this:

seafile@seafile-server:/tmp/log4j-scan# ./log4shell_1.3.1-log4shell_Linux_x86_64 scan /home/seafile/
11:21PM ??? Identified vulnerable path
cve: CVE-2021-44228
fileName: org/apache/logging/log4j/core/lookup/JndiLookup.class
hash: 0f038a1e0aa0aff76d66d1440c88a2b35a3d023ad8b2e3bac8e25a3208499f7e
path: /home/seafile/seafile-pro-server-8.0.12/pro/elasticsearch/lib/log4j-core-2.11.1.jar
severity: 10.0
versionInfo: “2.10.0, 2.11.0, 2.11.1, 2.11.2, 2.9.0, 2.9.1”

Further system check:

First, one should check the following log file “logs/elasticsearch.log” for the string “jndi”.

Furthermore, you should monitor the logging IPs and the system for unusual scripts. Unusual SSH public keys should also be removed. The following commands will help:

  • last -30 -i (shows the last logged in users)
  • netstat -tulpn (shows all services that open ports)
  • cat /${HOME}/.ssh/authorized_keys (shows the public keys)

important sources:

We will continue to inform you here.

Christoph Dyllick-Brenzinger
datamate GmbH & Co. KG


German version (letztes Update 17.12.2021, 1:30Uhr)

Hier der aktuelle Stand unserer Analysen zum Sicherheitsvorfall log4j - CVE-2021-44228:

Wichtig: das Thema ist dynamisch und muss aktiv weiter beobachtet werden. Aussagen, die heute noch aktuell sind, können morgen bereits nicht mehr gültig sein. Wir werden Sie aber auf jeden Fall weiterhin an dieser Stelle informieren.

Beschreibung der Sicherheitslücke:

Die Sicherheitslücke in log4j nutzt die ungewollte Verarbeitung von manipulierten log-Einträgen aus. Manipulierte Log-Einträge erlauben das Nachladen und Ausführen von Schadecode, der über das Internet heruntergeladen wird. Im schlimmsten Fall erlaubt diese Sicherheitslücke einem Angreifer sich root-Rechte auf dem System zu verschaffen und Code auszuführen, um sich z.B. eine Hintertür einzubauen.

Seafile Professional Edition (PE) der Version 7 und 8 nutzt für die Volltextsuche elasticsearch 5.6 gebundelt mit einer der Version 2.11 von log4j. Die Version 6 von Seafile PE verwendet elasticsearch 2.4.5 mit der Version 1.2 von log4j. Beide Versionen von log4j gelten mittlerweile als angreifbar. Somit ist ein Seafile Server grundsätzlich bedroht.

Bedrohungssituation für Seafile Server Professional Edition:

Nach aktueller Einschätzung ist die Gefahrenlage moderat. Der Angriff kann zwar nicht direkt durch einen URL-Aufruf ausgelöst werden, aber er kann über die Suche in veröffentlichten Bibliotheken ausgenutzt werden, d.h. auch ohne Benutzerkonto. Sind keine Bibliotheken veröffentlicht, kann ein Angriff nur durch einen eingeloggten User erfolgen, entweder durch eine manipulierte Datei oder eine Anfrage in der Volltextsuche.

Weiterhin gilt, dass Sie NICHT betroffen sind, wenn EINE der folgenden Bedingungen zugrifft:

  • Sie nutzen Seafile Server Community Edition (CE nutzt kein elasticsearch)
  • Sie haben die Volltextsuche in der seafevents.conf deaktiviert ([INDEX FILES] enabled = false)
  • Ihr Seafile-Server darf keinen Code aus dem Internet nachladen

Sollte keine dieser Bedingungen zutreffen, ist die Schwachstelle grundsätzlich ausnutzbar. Log-Einträge in der seahub.log weisen darauf hin, dass versucht wurde, diese Schwachstelle per HTTP-Request auszunutzen. Dies ist wie oben dargestellt nicht möglich.

Wichtige neue Erkenntnis:
Die folgenden zwei Bedingungen reichen doch nicht aus und bieten keinen vollständigen Schutz:

Ansatzpunkte um die Schwachstelle auszuhebeln

Die folgende Grafik hilft beim Verständnis der möglichen Mitigationen.

Mitigation für Seafile 7.0 und neuer:

Nach aktuellem Kenntnisstand gibt es nur zwei Möglichkeiten, um sich gegen die log4j Schwachstelle zu schützen:

A) die sicherste Variante ist es die Volltextsuche von Seafile zu deaktivieren. In diesem Fall werden keine Java-Prozesse ausgeführt und es kann kein Schadcode nachgeladen werden. Dies entspricht in der oberen Grafik den Fall 2 “DISABLE LOG4J”.

B) die zweite Möglichkeit besteht darin den kritischen JNDI Lookup Befehl zu patchen. Hierzu genügt es den folgenden Befehl im Verzeichnis /seafile-server-latest/pro/elasticsearch/lib/ auszuführen:

zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

Dies entspricht dem Punkt 3 in der Grafik “DISABLE JNDI LOOKUPS”

(Quelle: https://blog.cloudflare.com/inside-the-log4j2-vulnerability-cve-2021-44228/)

Dieser Befehl wurde von uns bereits auf mehreren Seafile 7.x und 8.x Versionen getestet, ohne die Funktionsweise von Seafile oder der Volltextsuche einzuschränken.

Mitigation für 6.3 und älter:

Wir empfehlen ein Update auf mindestens Seafile Server Professional Edition 7.1 oder neuer und den empfohlenen Ansatz für diese Version.

Scan nach verwundbaren log4j Klassen

LunaSec bieten auf Github ein Tool an, mit dem man nach verbundbaren log4j Klassen scannen kann (https://github.com/lunasec-io/lunasec/releases/). Mit den folgenden Befehlen können Sie den Scan selber auf Ihrem Seafile Server ausführen:

mkdir /tmp/log4jscan
cd /tmp/log4jscan
wget https://github.com/lunasec-io/lunasec/releases/download/v1.3.1-log4shell/log4shell_1.3.1-log4shell_Linux_x86_64
chmod +x log4sheel_1.3.1-log4shell_Linux_x86_64
./log4sheel_1.3.1-log4shell_Linux_x86_64 scan /home/seafile/seafile-server-latest/

Hinweise:

  • diese Befehle sind für einen 64-bit Linux Server. Passen Sie den Link im wget Befehl entsprechend an, wenn Sie ein anderes System haben
  • wenn Ihr Seafile nicht in /home/seafile/ installiert ist, passen Sie den Pfad entsprechend an
  • vergessen Sie den “/” am Ende des Pfades nicht. Ansonsten funktioniert der scan-Befehl nicht.

Das Ergebnis sieht dann z.B. so aus:

seafile@seafile-server:/tmp/log4j-scan# ./log4shell_1.3.1-log4shell_Linux_x86_64 scan /home/seafile/
11:21PM ??? Identified vulnerable path
cve: CVE-2021-44228
fileName: org/apache/logging/log4j/core/lookup/JndiLookup.class
hash: 0f038a1e0aa0aff76d66d1440c88a2b35a3d023ad8b2e3bac8e25a3208499f7e
path: /home/seafile/seafile-pro-server-8.0.12/pro/elasticsearch/lib/log4j-core-2.11.1.jar
severity: 10.0
versionInfo: “2.10.0, 2.11.0, 2.11.1, 2.11.2, 2.9.0, 2.9.1”

Weitere Systemprüfung:

Als erstes sollte man die folgende Log-Datei “logs/elasticsearch.log” auf den String “jndi” prüfen.

Weiterhin sollte man die sich eingeloggenden IPs und das System auf ungewöhnliche Skripte überwachen. Auch ungewöhnliche SSH Public Keys sollten entfernt werden. Die folgenden Befehle helfen hier weiter:

  • last -30 -i (zeigt die letzten eingeloggten User an)
  • netstat -tulpn (zeigt alle Dienste an, die Ports öffnen)
  • cat /${HOME}/.ssh/authorized_keys (zeigt die Public Keys an)

wichtige Quellen:

Wir werden Sie weiterhin hier informieren.

Christoph Dyllick-Brenzinger
datamate GmbH & Co. KG

5 Likes

You write, that the option “FILE_INDEXING = false” should be set in seafevents.conf. I don’t have this option set, because the manual says:

“Full text search is not enabled by default to save system resources.”

My pro-data directory is emtpy, so there is no file index, and I don’t have a elasticsearch.log.

Hey uosseafile,

if you don’t have an entry like the following in your seafevents.conf, then your fulltext search was / is disabled and you don’t have to worry. Consequently when full text search is disabled you have no elasticsearch.log. So what you describe makes sense.

[INDEX FILES]
## must be "true" to enable search
enabled = true

Does that clarify your question?

Best regards
Christoph

1 Like

Hi Christoph,

but there is a difference between

[INDEX FILES]
enabled = true

which enables or disables the file name search feature and

[INDEX FILES]
enabled = true
index_office_pdf = false

which enables file search (and therefore Elasticsearch) but disables full text search.

Only if index_office_pdf = false is set, full text search is disabled.

Dear Christoph, thanks for the writeup!
Still I am concerned because “heise.de” states that even if you prevent code from being loaded by setting the above mentioned java-option you can still be affected:

Warnstufe Rot: Log4j-Zero-Day-Lücke bedroht Heimanwender und Firmen | heise online (sorry, but I could not find an english version so far).

Please let me know what you think about this.

The question is, is only the full text search a security issue or the search feature at all ?

Also i do not know any setting which is named FILE_INDEXING in seafevents.conf ?

Hi @christophdb,

Thanks for your explanation and the update.

I can see that there have been multiple attempts on my seafile server. The attacks are in all cases done on the IP address of the server and not on the seahub/seafile domain virtualhost.

Visitors get a nginx 404 error when visiting the server directly at the IP address for HTTP and HTTPS connections. This means that the attempts were made on the server’s default virtualhost and not on the seafile virtualhost.

I cant find jndi in the elasticsearch and seafile virtualhost access logs.

You can use the following info to detect log4j exploitation & also do a grep in /logs/elasticsearch.log:

I think I’m fine based on these observations. In addition, i’ll apply this:

-Dlog4j2.formatMsgNoLookups=true

1 Like

@christophdb: Thank you very much for this in-depth post. I would like to add one comment: you do not necessarily need to be logged in to use the full text search. A published library also offers the same functionality without requiring any form of authentication.

2 Likes

Hey everybody,

first I have to apologize for “FILE_INDEXING = false”. There is no such parameter in seafevents.conf. The full text search is configured in seafevents.conf with the following parameters:

[INDEX FILES]
enabled = true
index_office_pdf = true

(more infos here: seafevents.conf - Seafile Admin Manual)

But from my point of view the second parameter “index_office_pdf” is not relevant at all. If “enabled = true” then elasticsearch is started. If “enabled = false” there is no elasticsearch process and consequently no log4j no matter what value index_office_pdf has.

I just tested that behaviour some minutes ago:

  • enabled = false
  • in controller.log:
  • 2021-12-13 13:39:20 seafile-controller.c(462): search is not enabled

or …

  • enabled = true
  • in controller.log
  • 2021-12-13 13:52:17 seafile-controller.c(96): spawn_process: /home/seafile/seafile-pro-server-8.0.12/pro/elasticsearch/bin/elasticsearch -Epath.logs=/home/seafile/logs -Epath.data=/home/seafile/pro-data/search/data -Enetwork.host=127.0.0.1 -p /home/seafile/pids/elasticsearch.pid

I hope this helps.
Still the recommended solution to fix this issue (even by BSI: https://www.bsi.bund.de/SharedDocs/Cybersicherheitswarnungen/DE/2021/2021-549032-10F2.pdf?__blob=publicationFile&v=3) is by adding the parameter “-Dlog4j2.formatMsgNoLookups=true” to jvm.options.

Best regards
Christoph

@daniel.pan

Hi,
can you confirm it is also safe on the indexing process side?
poi/ExtractText.jar and poi/log4j-1.2.13.jar ?

Regards.

The vulnerability impacts Apache Log4j 2 versions 2.0 to 2.14.1. So log4j-1.2.13.jar is not affected. And ExtractText does not produce logs.

At the beginning there was a little doubt about log4j 1.x versions, but now we know that it is safe without the JMS Appender component.
And as you said that ExtractText does not produce logs, it is OK.
So thanks for your answer!

Short update from the Elasticsearch site:

Elasticsearch 5 is susceptible to both remote code execution and an information leak via DNS. For versions 5.6.11 - 5.6.16, this can be mitigated by setting the JVM option . Users on an earlier version of 5.x, are recommended to upgrade to 5.6.16.

As seafile seems to use Elasticsearch 5.6.13, the mitigation by setting the JVM option provides provides full protection against the RCE and information leak attacks according to the Elasticsearch page.

I’m using the docker version of Seafile Pro. Where is the config file for elasticsearch located in this setup?

(already asked here: URGENT! Zero-Day-Exploit in log4j - #20 by Unreality)

Regards,
Martin Angermeier

We updated our summary at the top of this post (dez. 14th, 6.15pm)
Best regards
Christoph Dyllick-Brenzinger

3 Likes

I don’t have a Docker-based Seafile Server at hand, so I cannot verify, but I’d assume that the docker-compose.yml gives a good hint:

elasticsearch:
image: seafileltd/elasticsearch-with-ik:5.6.16
container_name: seafile-elasticsearch
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- “ES_JAVA_OPTS=-Xms1g -Xmx1g”
ulimits:
memlock:
soft: -1
hard: -1
mem_limit: 2g
volumes:
- /opt/seafile-elasticsearch/data:/usr/share/elasticsearch/data # Requested, specifies the path to Elasticsearch data persistent store.
networks:
- seafile-net

Have you checked the volume path?

Hi,

JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 are not affected by the LDAP attack vector.

In my case:(default ubuntu installation)

user@server:~# java -version
openjdk version “1.8.0_292”

Please note, this is LDAP only. There are also attacks using DNS and other protocols.

ldap[s]?|rmi|dns|nis|iiop|corba|nds|http

I think this is whats needed.

docker run \
...
    -e "ES_JAVA_OPTS=-Xms1g -Xmx1g -Dlog4j2.formatMsgNoLookups=true" 
...

Update from LunaSec:

The new CVE states:

It was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations.

We found this report concerning as it states that one of the recommended temporary mitigations for versions 2.7.0 <= Apache log4j <= 2.14.1 might not protect against this CVE. Further:

Note that previous mitigations involving configuration such as to set the system property log4j2.noFormatMsgLookup to true do NOT mitigate this specific vulnerability.

It was not readily apparently from reading this CVE what specifically was affected, or how this vulnerability is actually triggered. That’s why we actually tested this ourselves to figure out what the impact was.

Log4Shell Update: Second log4j Vulnerability Published (CVE-2021-44228 + CVE-2021-45046) | LunaSec

@daniel.pan Is it possible to update ElasticSearch asap with log4j 2.16?

For other pro users: Disable ElasticSearch with

[INDEX FILES]
enabled = false

4 Likes

We have just uploaded Seafile Pro 9.0.2, which we have tested internally for a few weeks. You can use ElasticSearch 6.x version with Seafile Pro 9.0.2. ElasticSearch 6.x is still in support pediod.

1 Like