====== Troubleshooting ======

===== Links =====

  * [[https://wiki.shibboleth.net/confluence/display/IDP4/Troubleshooting|Troubleshooting]] page in the official Shibboleth documentation

===== Where to download the metadata of your IdP/SP =====

Here is how you can get the metadata of your IdP or SP as they are currently published to the federation:
  * Log in to the metadata administration tool.
  * Select the Entity you are interested in and click the download icon in the actions column on the right side.

{{:en:metadata_admin_tool:metadata-download-en.png?800|}}

===== General troubleshooting procedure =====
  * Watch the log file ''idp-process.log'' (default path: ''/opt/shibboleth-idp/logs'') and the Tomcat log.
  * Increase the log level to DEBUG. While you can see warnings and errors without doing so, the content of SAML assertions stays hidden without the DEBUG log level. In most cases it is sufficient to set DEBUG for idp, messages and encryption.<file xml /opt/shibboleth-idp/conf/idp.properties>idp.loglevel.idp = DEBUG
idp.loglevel.messages = DEBUG
idp.loglevel.encryption = DEBUG</file>
  * Trigger a configuration reload manually (''touch /opt/shibboleth-idp/war/idp.war'') or wait for the automatic reload of the logging configuration. The reload intervals can be adapted in ''/opt/shibboleth-idp/conf/services.properties''.
  * If you see any warnings or errors in the idp-process.log, read the log file from top to bottom beginning at the last restart of the IdP. This is important because you can mostly find the reason (like syntax errors in configuration files incl. the line number) in the first messages, whereas further below you only get consequential errors. They tend to tell little about where to dig further like in this example:<code>'shibboleth.metrics.RegisterMetricSets$child#0' defined in file [C:\Program Files (x86)\shibboleth\idp\system\conf\..\..\conf\admin\metrics.xml]: Cannot resolve reference to bean 'shibboleth.metrics.AttributeResolverGaugeSet' while setting bean property 'arguments' with key [7]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'shibboleth.metrics.AttributeResolverGaugeSet' defined in file [C:\Program Files (x86)\shibboleth\idp\system\conf\general-admin-system.xml]: Invocation of init method failed; nested exception is net.shibboleth.utilities.java.support.component.ComponentInitializationException: Injected service was null or not an AttributeResolver</code>Scroll up and search for errors that occur during the parsing of ''conf/attribute-resolver.xml''.
  * When editing the values in ''.properties'' files, please avoid trailing spaces at the line ends. A blank space at the end of a password renders the password unusable. If you get the LDAP error ''Invalid credentials'', make sure that your credentials are correct and **check for spaces at the line ends of the credentials**. The password must not be written in between single or double quotes.
  * Be precise when editing configuration files. An example: Entity IDs with or without a trailing slash are two different Entity IDs!

===== The IdP does not write any log files =====
If your IdP does not log anything at all to the files in ''/opt/shibboleth-idp/logs'' (or any custom directory you configured), please check the Tomcat log files. On a Debian server they are located in ''/var/log/tomcat9''. Watch //all// of them, e.g. with<code bash>root@idp:~# tail -f /var/log/tomcat9/*.log</code> and restart Tomcat in a different terminal window.

===== Which attributes and attribute values were transmitted to an SP? =====

To see the full SAML assertion, set the log level to DEBUG for idp, messages and encryption as shown above. The SAML assertion will then be written into ''idp-process.log''. Search for the string **"Assertion before encryption"**. In this example of a SAML assertion you can see that the IdP with the Entity ID https://idp.local/idp/shibboleth transmitted a persistentID (value: "NMTR6SVZOCUWF5MNGDDIYQQBY4QCB76N") as well as the attributes ''eduPersonScopedAffiliation'' (with three values) and ''eduPersonEntitlement'' (with one value) to an SP with the Entity Id https://sp1.local/shibboleth.

<file xml /opt/shibboleth-idp/logs/idp-process.log>
2020-10-27 14:54:07,079 - 127.0.0.2 - DEBUG [org.opensaml.saml.saml2.encryption.Encrypter:339] - Assertion before encryption:
<?xml version="1.0" encoding="UTF-8"?><saml2:Assertion xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion" ID="_3a89275f3cd179685b22a3cf616c518a" IssueInstant="2020-10-27T13:54:06.179Z" Version="2.0">
   <saml2:Issuer>https://idp.local/idp/shibboleth</saml2:Issuer>
   <saml2:Subject>
      <saml2:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" NameQualifier="https://idp.local/idp/shibboleth" SPNameQualifier="https://sp1.local/shibboleth" xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion">NMTR6SVZOCUWF5MNGDDIYQQBY4QCB76N</saml2:NameID>
      <saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
         <saml2:SubjectConfirmationData Address="127.0.0.2" InResponseTo="_72e4da8e1fa8dfcea0f85a340ded8b7c" NotOnOrAfter="2020-10-27T13:59:06.581Z" Recipient="https://sp1.local/Shibboleth.sso/SAML2/POST"/>
      </saml2:SubjectConfirmation>
   </saml2:Subject>
   <saml2:Conditions NotBefore="2020-10-27T13:54:06.179Z" NotOnOrAfter="2020-10-27T13:59:06.179Z">
      <saml2:AudienceRestriction>
         <saml2:Audience>https://sp1.local/shibboleth</saml2:Audience>
      </saml2:AudienceRestriction>
   </saml2:Conditions>
   <saml2:AuthnStatement AuthnInstant="2020-10-27T13:53:52.456Z" SessionIndex="_d8a534b987abc20fe007a9cc80af7bdb">
      <saml2:SubjectLocality Address="127.0.0.2"/>
      <saml2:AuthnContext>
         <saml2:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport</saml2:AuthnContextClassRef>
      </saml2:AuthnContext>
   </saml2:AuthnStatement>
   <saml2:AttributeStatement>
      <saml2:Attribute FriendlyName="eduPersonScopedAffiliation" Name="urn:oid:1.3.6.1.4.1.5923.1.1.1.9" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
         <saml2:AttributeValue>staff@local</saml2:AttributeValue>
         <saml2:AttributeValue>member@local</saml2:AttributeValue>
         <saml2:AttributeValue>employee@local</saml2:AttributeValue>
      </saml2:Attribute>
      <saml2:Attribute FriendlyName="eduPersonEntitlement" Name="urn:oid:1.3.6.1.4.1.5923.1.1.1.7" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
         <saml2:AttributeValue>urn:mace:dir:entitlement:common-lib-terms</saml2:AttributeValue>
      </saml2:Attribute>
   </saml2:AttributeStatement>
</saml2:Assertion>
</file>

===== Attribute samlPairwiseID does not have any transcoding rules =====
Shibboleth IdPs 4.x that were **upgraded from a version 3.x** do not automatically use the attribute registry. The configuration file ''attribute-Resolver.xml'' then still contains the transcoding rules for each defined attribute. When adding a new attribute definition you also have to specify a transcoding rule, or else the IdP will not be able to encode the attribute in any SP-readable form. Here is an example:<file xml /opt/shibboleth-idp/conf/attribute-resolver.xml>
    <AttributeDefinition xsi:type="Scoped" id="samlPairwiseID" scope="%{idp.scope}">
        <InputDataConnector ref="myStoredId" attributeNames="persistentId"/>
        <AttributeEncoder xsi:type="SAML2ScopedString" name="urn:oasis:names:tc:SAML:attribute:pairwise-id" friendlyName="pairwise-id" encodeType="false" />
    </AttributeDefinition>
</file>
Alternatively, you can reconfigure the IdP to use the attribute registry [[de:shibidp:upgrade#umstellung_auf_die_nutzung_der_attribute_registry_optional|(documentation in German)]].

Note that the ''AttributeEncoder'' line must not be included in the attribute definition of a **newly installed IdP 4.x**. It will be fetched from the attribute registry underneath ''conf/attributes'' instead.


===== opensaml::SecurityPolicyException =====

"opensaml::SecurityPolicyException Message was signed, but signature could not be verified."

You see this error message whenever the IdP certificate published in the federation metadata does not match the one configured on the actual IdP. During installation the Shibboleth installer generates a self-signed certificate and preconfigures it in ''conf/idp.properties''. Adapt that file to point to the certificate you want the IdP to use and make sure the same one is published.

By the way: The file ''metadata/idp-metadata.xml'' is autogenerated, too.  It contains the initial post-installation IdP metadata. It is parsed when you first add the IdP to the metadata administration tool, but in the actual federation this file is ignored. The valid IdP metadata that you maintain are those in the administration tool.

===== opensaml::FatalProfileException =====
"opensaml::FatalProfileException at (https://testsp2.aai.dfn.de/Shibboleth.sso/SAML2/POST)"

You get this error message when **the Service Provider cannot find any metadata for the Identity Provider**.

  * Check if you have added the IdP to the metadata administration tool and if it was added to the respective federation correctly (DFN-AAI-Test, DFN-AAI-Basic, or DFN-AAI).
  * Compare the Entity ID in ''conf/idp.properties'' with the one in the metadata entry. They have to be identical.
  * After a change to the federation metadata, keep in mind that you have to wait for 60-90 minutes for the metadata to be aggregated and redistributed to all SPs.

===== No metadata returned =====
When you see the following message in your ''idp-process.log'', **the Identity Provider cannot find any metadata for the Service Provider**:<code>INFO [org.opensaml.saml.common.binding.impl.SAMLMetadataLookupHandler:171] (...) Message Handler:  No metadata returned for 
 (...) in role {urn:oasis:names:tc:SAML:2.0:metadata}SPSSODescriptor</code>

  * Please check where the IdP should know the SP from:
    * from federation metadata?
    * from your organisation's [[en:metadata_local|local metadata]]? (Did you add the SP as a local SP in the metadata administration tool?)
    * from an xml metadata file that you added manually to ''conf/metadata-providers.xml''? Check the content of both files.
  * Remember that - in the first two cases - you have to wait for 60 to 90 minutes for the changes to propagate.

===== The application you have accessed is not registered for use with this service =====

"Web Login Service - Unsupported Request The application you have accessed is not registered for use with this service", or in German: "Web Anmeldedienst - Nicht unterstützte Anfrage Die Applikation, auf die Sie zugreifen möchten, ist für die Benutzung dieses Dienstes nicht registriert."

This error message is displayed when the IdP cannot find the SP in metadata.
  * Check whether all required metadata providers have been added to ''conf/metadata-providers.xml'' ([[de:shibidp:config-metadata|Documentation]]).
  * Check the folder ''/opt/shibboleth-idp/metadata'' to see if up-to-date federation metadata have been downloaded. You can also access this information on the IdP status page (default: https://YOUR-HOST/idp/status). In the following example DFN_AAI metadata have expired:<file>service: shibboleth.MetadataResolverService
last successful reload attempt: 2020-12-22T07:58:12Z
last reload attempt: 2020-12-22T07:58:12Z

	metadata source: DFN_AAI
	last refresh attempt: 2020-12-24T05:26:48Z
	last update: 2020-12-24T05:26:48Z

	metadata source: DFN_AAI_eduGAIN
	last refresh attempt: 2021-01-05T08:57:55Z
	last update: 2021-01-05T08:57:55Z

	metadata source: DFN_AAI_TEST
	last refresh attempt: 2021-01-05T09:36:13Z
	last update: 2021-01-05T09:36:13Z
</file>
  * Check the IdP's DEBUG-Log. Compare the saml:Issuer from the AuthnRequest with the EntityID you are trying to contact. If there is a different issuer string in the Authentication Request the IdP cannot find the issuer in the federation metadata. Contact the SP operator in this case.

===== DecryptNameIDFailed =====
If you see the error message "A non-proceed event occurred while processing the request: DecryptNameIDFailed" this might be after a certificate rollover and maybe it only occurs upon Single Logout. The Service Provider has encrypted data with a certificate for which the IdP does not have a private key (any more). Remove the old certificate from the metadata and wait for 60 to 90 minutes. By then, the information to only use the new certificate has propagated.

===== Reset a configuration file to default =====
Your IdP keeps copies of all original files in the folder ''dist/conf''. Here you can fetch files, if you want to start over or if you want to see what the original file looks like in the latest installed IdP version.

===== Duplicate attributes in Shibboleth IdP 4.x =====
If you notice that your IdP 4.x transmits duplicate attributes, you probably have copied the file ''conf/attribute-resolver.xml'' from an IdP 3.x without adapting it. The IdP has duplicate transcoding rules: once in ''conf/attribute-resolver.xml'' and once in the Attribute Registry. Remove the Attribute Encoder lines from the resolver configuration so that it looks like in [[de:shibidp:config-attributes-minimal|this example]].

===== Duplicate Transcoding Rule =====
If you get the error message below, you probably have a duplicate attribute in your Attribute Registry. Maybe you imported attributes from a file like our dfnMisc.xml ([[de:shibidp:config-attributes#edupersontargetedid_und_andere_verbreitete_attribute_hinterlegen|German documentation]]) **and** you have defined one of the attributes in an individual .properties file underneath ''conf/attributes/custom/''? Make sure that each attribute exists in the Attribute Registry exactly once.

<code bash>java.lang.IllegalArgumentException: {urn:oasis:names:tc:SAML:2.0:assertion}NameID is
already the child of another XMLObject and may not be inserted into this list</code>

===== IdP/SP is no longer part of the eduGAIN metadata =====

Our downstream eduGAIN metadata (the eduGAIN metadata we distribute to DFN-AAI) have never contained entities from DFN-AAI. We filter them out because your systems already know them from DFN-AAI metadata and we do not want to distribute duplicates. To check whether an entity is part of the eduGAIN metadata, please search for it in the [[https://technical.edugain.org/entities|eduGAIN Entities Database]].

===== IdP is not displayed in Discovery Services =====

You have added your Identity Provider to the federation but it doesn't show in discovery services? This can happen for several reasons:

  * **The Service Provider hasn't fetched the latest metadata yet.** Please wait for 60-90 minutes before testing in DFN-AAI or our test federation). Wait for up to 24 hours before testing in eduGAIN. You can check if your IdP is already included in the eduGAIN metadata: https://technical.edugain.org/entities.
  * You have ticked the checkbox "hide from discovery" in the IdP's settings in the metadata administration tool. Remove the tick and wait for 60-90 minutes.
  
===== SP Metadata: AuthnRequestsSigned and WantAssertionsSigned =====

A Service Provider can announce in its metadata that it
  * signs Authentication Requests it sends to IdPs, and/or
  * wants to get signed SAML assertions back.

Our metadata administration tool only displays this information if it is included in the xml files upon initial upload to the metadata administration. Please extend your SP metadata like this:<code xml><md:SPSSODescriptor AuthnRequestsSigned="true" WantAssertionsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol"></code>
    
{{tag>idp4 troubleshooting debugging debug logging}}