Skip to content

[Bug]: CalDAV free/busy request on user's outbox collection always returns 3.7;Could not find principal #50239

@mcrha

Description

@mcrha

⚠️ This issue respects the following points: ⚠️

  • This is a bug, not a question or a configuration/webserver/proxy issue.
    This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
    Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
    I agree to follow Nextcloud's Code of Conduct.

Bug description

Coming from downstream
https://gitlab.gnome.org/GNOME/evolution-data-server/-/issues/467

The free/busy requests over CalDAV always return 3.7;Could not find principal, even when asking for the logged in user

Steps to reproduce

Send a request to the server, to the user's scheduling Outbox, which is used for free/busy lookup:

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
METHOD:REQUEST
BEGIN:VFREEBUSY
UID:5328b732-5010-47b3-8bac-9bb36c2e51b8
DTSTAMP:20250117T114144Z
DTSTART:20250117T070000Z
DTEND:20250202T070000Z
ORGANIZER:mailto:organizer@no.where
ATTENDEE;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;CUTYPE=INDIVIDUAL:mailt
 o:user@no.where
END:VFREEBUSY
END:VCALENDAR

The server responds:

<?xml version="1.0" encoding="utf-8"?>
<cal:schedule-response xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/">
  <cal:response>
    <cal:recipient>
      <d:href>mailto:user@no.where</d:href>
    </cal:recipient>
    <cal:request-status>3.7;Could not find principal</cal:request-status>
  </cal:response>
</cal:schedule-response>

regardless the attendee is or is not a legitimate user configured on the server.

Expected behavior

The user is found, and its free/busy information is returned from the server.

It seems searchPrincipals() function is called with principals $prefixPath, which is not known to the function itself. When I add it beside the principals/users case, then it can find the user, but it fails later on. The change looks like:

--- nextcloud/apps/dav/lib/Connector/Sabre/Principal.php.orig	2019-05-15 10:39:32.000000000 -0400
+++ nextcloud/apps/dav/lib/Connector/Sabre/Principal.php	2025-01-17 09:51:10.089227728 -0500
@@ -331,6 +331,7 @@ class Principal implements BackendInterf
 		}
 
 		switch ($prefixPath) {
+			case 'principals':
 			case 'principals/users':
 				return $this->searchUserPrincipals($searchProperties, $test);

More details can be found in the downstream bug.

I do not know how to figure out the exact server version from the web UI, the "About" menu opens some marketing stuff, not a version info like in other apps, thus I cannot tell what precise version this is, I'm sorry, but looking into the latest code the above change is still applicable.

Nextcloud Server version

28

Operating system

None

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
    LDAP/ Active Directory
    SSO - SAML
    Other

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

No response

Activity

added
0. Needs triagePending check for reproducibility or if it fits our roadmap
on Jan 17, 2025
kesselb

kesselb commented on Jan 18, 2025

@kesselb
Collaborator
st3iny

st3iny commented on Feb 3, 2025

@st3iny
Member

Hi, thanks for the report.

Does a user with the email user@no.where exist on the server? Is this email address the primary email address of the user? Please have a look a the personal settings of the user.

EDIT: Correct the email address.

mcrha

mcrha commented on Feb 3, 2025

@mcrha
Author

I changed the address for privacy reason in the description. It fails also for the user itself, the one being logged in. It's the only email address the user has set, thus might be the primary.

If you re-read the description, you'll see there is a block with a patch there - the function the patch touches never tries to search for the user, due to the parameter being different than expected. If I correct it (apply the patch), it fails later on, which was beyond my abilities to investigate it further.

Maybe for easier testing, the client runs this query over https::

POST /remote.php/dav/calendars/mcrha/outbox/ HTTP/1.1
Content-Type: text/calendar; charset="utf-8"

BEGIN:VCALENDAR
CALSCALE:GREGORIAN
PRODID:-//Ximian//NONSGML Evolution Calendar//EN
VERSION:2.0
METHOD:REQUEST
BEGIN:VFREEBUSY
UID:fa2566720b5589ed27115df0ec3b16dfaf098b61
DTSTAMP:20250203T164701Z
DTSTART:20250126T230000Z
DTEND:20250302T230000Z
ORGANIZER:mailto:mcrha
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION;RSVP=FALSE:mailto:user@no.where
END:VFREEBUSY
END:VCALENDAR

and the server responds:

HTTP/1.1 403 Forbidden
Content-Type: application/xml; charset=utf-8
Server: nginx

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
	<s:exception>Internal Server Error</s:exception>
	<s:message>
		The server was unable to complete your request.		If this happens again, please send the technical details below to the server administrator.		More details can be found in the server log.			</s:message>

	<s:technical-details>
		<s:remote-address>0.0.0.0</s:remote-address>
		<s:request-id>aGIOMI3gtGtnUmzi6iL3</s:request-id>

		</s:technical-details>
mcrha

mcrha commented on Feb 3, 2025

@mcrha
Author

Okay, I see the response is slightly different than in the description. It's what I get when I apply the patch. I re-tried with a newer Nexcloud (but I do not know where to find the exact version; if you tell me where it is in the web interface, then I can paste it here), which might have the part with the patch changed already or something, I cannot tell for sure, because I do not have access to the server itself. The line is 401 for the patch at the moment, inside the searchPrincipals() function.

I verified the logged-in user's email address is the one I used in the ATTENDEE property of the component, thus there's everything as it should be.

Mer0me

Mer0me commented on Jun 4, 2025

@Mer0me

Recently updated to Nextcloud 31 on an instance with ~600 users. Only some users' free/busy information could be fetched since update. It used to work on Nextcloud 30.
I was able to fix the problem for us with the info on this issue (thanks to @mcrha ).

Most of our users have their email address in oc_preferences table stored with upper and lowercase chars. LDAP / Active Directory users certainly get their email address from the AD email field and it's like "John.DOE@domain.com"

On the file lib/private/AllConfig.php, line 406, there is a function getUsersForUserValueCaseInsensitive which claims in his name to perform a case insensitive search, but it doesn't ! On line 412 (edit : my mistake, it's not exactly here, see my post further), the searchUsersByValueString function is called omitting the last argument, and this last argument is the one switching to a case sensitive or insensitive search. By default, it is set to false in the lib/private/Config/UserConfig.php file (line 409). Omitting the argument, the search is case sensitive, and all of our users having uppercase chars in their email address on the oc_preferences tables can't be fetched by the function, returning 3.7;Could not find principal

I don't know how to propose this modification to the community (I'm not familiar with github), so if someone could do this for me :

Line 412 of lib/private/AllConfig.php:

//$result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value));
$result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value, true));

I'm not sure this modification will fix the OP problem. But it's working for us.

st3iny

st3iny commented on Jun 4, 2025

@st3iny
Member

@Mer0me Thanks for the investigation. This already seems to be the case on the master and stable31 branches.

st3iny

st3iny commented on Jun 4, 2025

@st3iny
Member

@mcrha I have another question for you to debug this further: Could it be that there are multiple users on your system sharing the same email address?

You can find out quickly by running the following shell script:

echo 'select data from oc_accounts;' | mysql [...] | tail -n +2 | jq -r '.email.value | select(. != "") | select(. != null)' | sort | uniq -d

Mer0me

Mer0me commented on Jun 4, 2025

@Mer0me

@Mer0me Thanks for the investigation. This already seems to be the case on the master and stable31 branches.

My mistake. The omitted argument is in the getUsersForUserValue function, line 392 :

    public function getUsersForUserValue($appName, $key, $value) {
            /** @var list<string> $result */
           //$result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value));
            $result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value), true);
            return $result;
    }`

`

mcrha

mcrha commented on Jun 4, 2025

@mcrha
Author

That mysql [...] command for a locally running server precisely looks like what, please? It's a long time I played with the mysql database.

I do not think there are multiple users with the same email address, but I'll be happy to verify & confirm that.

Mer0me

Mer0me commented on Jun 13, 2025

@Mer0me

On v31.0.6, the problem mentioned on #50239 (comment) is still present

st3iny

st3iny commented on Jun 19, 2025

@st3iny
Member

@Mer0me Email addresses should always be saved lowercase. The logic indeed breaks down if some email addresses are mixed case.

Would it be possible for you to try the following patch? It removes the casing check.

diff --git a/lib/private/AllConfig.php b/lib/private/AllConfig.php
index 72af6c960a5..ff12cff9be9 100644
--- a/lib/private/AllConfig.php
+++ b/lib/private/AllConfig.php
@@ -404,10 +404,6 @@ class AllConfig implements IConfig {
 	 * @deprecated 31.0.0 - use {@see IUserConfig::searchUsersByValueString} directly
 	 */
 	public function getUsersForUserValueCaseInsensitive($appName, $key, $value) {
-		if ($appName === 'settings' && $key === 'email') {
-			return $this->getUsersForUserValue($appName, $key, strtolower($value));
-		}
-
 		/** @var list<string> $result */
 		$result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value, true));
 		return $result;
Mer0me

Mer0me commented on Jun 19, 2025

@Mer0me

Thanks, I'll try this patch tomorrow.
If the email addresses are mixed case, it's certainly because they've been imported directly as they are from Active Directory with Nextcloud LDAP connector. If Nextcloud is designed to use only lowercase email addresses, maybe the LDAP connector should be checked too ?

st3iny

st3iny commented on Jun 19, 2025

@st3iny
Member

I also created a PR here: #53608

It seems like the email addresses are taken as is from LDAP which is not good. We still need a migration to fix all existing users though.

Mer0me

Mer0me commented on Jun 20, 2025

@Mer0me

@Mer0me Email addresses should always be saved lowercase. The logic indeed breaks down if some email addresses are mixed case.

Would it be possible for you to try the following patch? It removes the casing check.

diff --git a/lib/private/AllConfig.php b/lib/private/AllConfig.php
index 72af6c960a5..ff12cff9be9 100644
--- a/lib/private/AllConfig.php
+++ b/lib/private/AllConfig.php
@@ -404,10 +404,6 @@ class AllConfig implements IConfig {

  • @deprecated 31.0.0 - use {@see IUserConfig::searchUsersByValueString} directly
    */
    public function getUsersForUserValueCaseInsensitive($appName, $key, $value) {
  • if ($appName === 'settings' && $key === 'email') {
    
  • 	return $this->getUsersForUserValue($appName, $key, strtolower($value));
    
  • }
    
  • /** @var list<string> $result */
    $result = iterator_to_array(\OCP\Server::get(IUserConfig::class)->searchUsersByValueString($appName, $key, $value, true));
    return $result;
    

It works for us. Thanks.

st3iny

st3iny commented on Jun 20, 2025

@st3iny
Member

Thanks for testing. I'll inquire internally if we can accept this fix. I suspect it might cause negative performance regressions.

st3iny

st3iny commented on Jun 20, 2025

@st3iny
Member

@mcrha Please try the patch as well. Feedback would be greatly appreciated to see if you are facing the same issue with mixed case email addresses.

mcrha

mcrha commented on Jun 20, 2025

@mcrha
Author

My problem was not about case sensitivity, but about unhandled case (see the "patch" in the description)

st3iny

st3iny commented on Jun 20, 2025

@st3iny
Member

Thanks for confirming. So we are talking about two different issues here.

st3iny

st3iny commented on Jun 20, 2025

@st3iny
Member

I still can't reproduce the initial problem. I tried latest master, stable31 and stable30. I posted the given Free/Busy ICS file (with another set of email addresses though).

Reply looked like this:

<?xml version="1.0" encoding="utf-8"?>
<cal:schedule-response xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
  <cal:response>
    <cal:recipient>
      <d:href>mailto:debug@imap.localhost</d:href>
    </cal:recipient>
    <cal:request-status>2.0;Success</cal:request-status>
    <cal:calendar-data>BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Sabre//Sabre VObject 4.5.6//EN
CALSCALE:GREGORIAN
METHOD:REPLY
BEGIN:VFREEBUSY
DTSTART:20250117T070000Z
DTEND:20250202T070000Z
DTSTAMP:20250620T085035Z
ATTENDEE:mailto:debug@imap.localhost
UID:5328b732-5010-47b3-8bac-9bb36c2e51b8
ORGANIZER:mailto:debug@imap.localhost
END:VFREEBUSY
END:VCALENDAR
</cal:calendar-data>
  </cal:response>
</cal:schedule-response>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @st3iny@kesselb@Mer0me@szaimen@mcrha

      Issue actions

        [Bug]: CalDAV free/busy request on user's `outbox` collection always returns `3.7;Could not find principal` · Issue #50239 · nextcloud/server