[Bug]: External Storage against BSD ftp server failing due to parsing difference

[kevemueller]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

When using external files against an FTP server running BSD (e.g. FreeBSD), the file listing functionality chokes on the output of the directory listing of the FTP server.
BSD ftpd prepends the ls -lgA listing with a line reading "total 123" (arbitrary number).
The parser is prepared to distinguish Windows / Unix, but not to skip over this line and throws an Exception:

Metadata can't be parsed from item 'total 17' , not enough parts.

Server side fixing of this is hard, as it is a built-in function compiled into ftpd and not a call to a bin/ls binary which could be tweaked.
Quick hack modification of the source using $filteredList = array_slice($rawList, 1) confirms that the remainder of the file listing is parsed properly and the expected functionality is available.

Steps to reproduce

1. Set-up a FTP server on FreeBSD or find a public one.
2. Set-up external files in Nextcloud to connect to said FTP server.
3. Browse the folder in the WebUI

Expected behavior

The external files FTP module should be smart enough to handle the output of a BSD ftpd server by checking the first line to be matching ^total.*$ and skipping over it.

Installation method

Other Community project

Operating system

Other

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "apps_paths": [
            {
                "path": "\/usr\/local\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            },
            {
                "path": "\/usr\/local\/www\/nextcloud\/apps-pkg",
                "url": "\/apps-pkg",
                "writable": false
            }
        ],
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "25.0.3.2",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "auth.bruteforce.protection.enabled": false,
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "proxy": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 2,
        "default_phone_region": "***REMOVED SENSITIVE VALUE***",
        "tempdirectory": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.17.0
  - bruteforcesettings: 2.5.0
  - calendar: 4.2.4
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - comments: 1.15.0
  - contacts: 5.1.0
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.1
  - files_external: 1.17.0
  - files_pdfviewer: 2.6.0
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - firstrunwizard: 2.14.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - oauth2: 1.13.0
  - password_policy: 1.15.0
  - photos: 2.0.1
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - recommendations: 1.4.0
  - related_resources: 1.0.3
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - sharebymail: 1.15.0
  - spreed: 15.0.4
  - support: 1.8.0
  - survey_client: 1.13.0
  - systemtags: 1.15.0
  - text: 3.6.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - weather_status: 1.5.0
  - workflowengine: 2.7.0
Disabled:
  - admin_audit
  - encryption
  - suspicious_login
  - twofactor_totp
  - user_ldap

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"ksvSoVltakmNXHVaeGOH", [...], "message":"Exception while scanning storage \"ftp::ftp@localhost//Scans\"",[...],"version":"25.0.3.2","exception":{"Exception":"RuntimeException","Message":"Metadata can't be parsed from item 'total 17' , not enough parts.","Code":0,"Trace":[{"file":"/usr/local/www/nextcloud/apps/files_external/lib/Lib/Storage/FtpConnection.php","line":144,"function":"parseUnixItem","class":"OCA\\Files_External\\Lib\\Storage\\FtpConnection","type":"->"},
[...]
}

Additional info

No response

[joshtrichards]

Probably needs to be changed to skip things right around here:

server/apps/files_external/lib/Lib/Storage/FtpConnection.php

Lines 135 to 140 in 5ed48e9

	private function parseUnixItem(string $item, string $directory): array {
	$item = preg_replace('#\s+#', ' ', $item, 7);
	if (count(explode(' ', $item, 9)) !== 9) {
	throw new \RuntimeException("Metadata can't be parsed from item '$item' , not enough parts.");
	}

Similar to what is done in the original implementation we were inspired here it looks like already:

https://github.com/thephpleague/flysystem/blob/235ae805a26f6fbd4d7a5f1c8e3bdf2c386adab3/src/Ftp/FtpAdapter.php#L365-L381

Feel up to trying to come up with a working solution and submitting it as a PR?