tag:blogger.com,1999:blog-78081172964919240452024-03-19T10:17:20.800+01:00Magnus K KarlssonI'm dedicated agile security architect/system architect/developer with specialty of open source framework.Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.comBlogger641125tag:blogger.com,1999:blog-7808117296491924045.post-63060036241940579792023-09-10T20:19:00.005+02:002023-09-10T22:24:17.150+02:00Keycloak, User Federation from LDAP and Lesson Learned<h1>LDAP</h1>
<p>We are going to use OpenLDAP and setup in previous blog <a href="https://magnus-k-karlsson.blogspot.com/2023/09/openldap-image-and-custom-ldif-with.html">OpenLDAP Image and Custom LDIF with User and Group</a>.</p>
<pre><code class="bash"
>$ podman run -d --name openldap \
-e LDAP_ROOT=dc=magnuskkarlsson,dc=se \
-e LDAP_ADMIN_USERNAME=admin \
-e LDAP_ADMIN_PASSWORD=changeit \
-e LDAP_CONFIG_ADMIN_ENABLED=true \
-e LDAP_ALLOW_ANON_BINDING=false \
-e LDAP_CUSTOM_LDIF_DIR=/ldifs \
-p 1389:1389 \
-p 1636:1636 \
-v ./ldifs:/ldifs:Z \
docker.io/bitnami/openldap:2.6
</code></pre>
<p>Test it is working.</p>
<pre><code class="bash"
>$ ldapsearch -H ldap://localhost:1389 -D cn=admin,dc=magnuskkarlsson,dc=se -w changeit -b dc=magnuskkarlsson,dc=se -s sub
</code></pre>
<h1>Keycloak Installation</h1>
<p>Here we will use the supported Keycloak version - RH SSO.</p>
<pre><code class="bash"
>$ unzip rh-sso-7.6.0-server-dist.zip
$ mv rh-sso-7.6 rh-sso-7.6.0
$ cd rh-sso-7.6.0/bin/
$ ./add-user-keycloak.sh --user admin --password admin
</code></pre>
<p><a href="https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.6/html-single/server_administration_guide/index#ldap_troubleshooting">Configure LDAP logging in RH SSO</a></p>
<pre><code class="bash"
>$ vim rh-sso-7.6.0/standalone/configuration/standalone.xml
...
<profile>
<subsystem xmlns="urn:jboss:domain:logging:8.0">
...
</logger>
<logger category="org.keycloak.storage.ldap">
<level name="TRACE"/>
</logger>
<root-logger>
...
</code></pre>
<p>Start</p>
<pre><code class="bash"
>$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk
$ ./standalone.sh -Djboss.socket.binding.port-offset=100
</code></pre>
<h1>Kecloak Configuration User Federation with NO Import</h1>
<p>We will start federate openldap user and roles and not import them and test what happens, to understand how user federation works.</p>
<p>Add User federation</p>
<p><b>Edit mode: READ_ONLY</b></p>
<p><b>Import Users: OFF</b></p>
<p><b>No Synchronization</b></p>
<p>Add Mappers:</p>
<ul>
<li>password: user-attribute-ldap-mapper; password; userPassword</li>
<li>role-ldap-mapper: role-ldap-mapper;</li>
</ul>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipM5nkaqNapfkdN-cJFcSpJMbzqE_PsSd8DZzbvyNONNQjdiI1OQfx3n2JRU5LPHfc4ZMHg7uHQEEyo537B7m0jzyEKY2Uy-wVIiFObh5aLoQpYfJw7BPWDFBYa35dl19-AEg39ra9atRqJdtuxwa8MsumH7PWzYC25AhW1V3VyTVIrM_8ytS6Fea2scI/s2169/Screenshot%20from%202023-09-09%2014-15-33.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="400" data-original-height="2169" data-original-width="1455" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipM5nkaqNapfkdN-cJFcSpJMbzqE_PsSd8DZzbvyNONNQjdiI1OQfx3n2JRU5LPHfc4ZMHg7uHQEEyo537B7m0jzyEKY2Uy-wVIiFObh5aLoQpYfJw7BPWDFBYa35dl19-AEg39ra9atRqJdtuxwa8MsumH7PWzYC25AhW1V3VyTVIrM_8ytS6Fea2scI/s400/Screenshot%20from%202023-09-09%2014-15-33.png"/></a></div>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGOsBOJcHD8z_EDt73KMHhpt0gq7EE7p-gzDi8nXN3S3etzhqB4vcw6_ytTyBXl06V8SXNZsgKN08uwuTsru1KLFKkU0kCX7xQW2W3D-NFgjWF_ZTAVt5TKpm-_q_q3sKV-V0OcOtAviNMczQn9EkWkBNokKIW9BcFRYxWAf20aDnHFRT-mugvQyIfiDk/s1455/Screenshot%20from%202023-09-09%2014-17-46.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="791" data-original-width="1455" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGOsBOJcHD8z_EDt73KMHhpt0gq7EE7p-gzDi8nXN3S3etzhqB4vcw6_ytTyBXl06V8SXNZsgKN08uwuTsru1KLFKkU0kCX7xQW2W3D-NFgjWF_ZTAVt5TKpm-_q_q3sKV-V0OcOtAviNMczQn9EkWkBNokKIW9BcFRYxWAf20aDnHFRT-mugvQyIfiDk/s320/Screenshot%20from%202023-09-09%2014-17-46.png"/></a></div>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZTrMaIoDRX15ztZS-oo5RIqVMHOBlGD2UCkgaCNTan091GUkDg9iaZqFjKFSfXKPolPkk6i1sWPYYnQdTMRjYxY1I2i7-_XuEklOTHF49sXOZZ9AfuSNhLcyY0hTHDYAX4nSa2z0MyFIzJw-R0glmB8kgi_sDTPv-dRmrxLIhiOInhbK7EWHzceLNC3I/s1523/Screenshot%20from%202023-09-09%2014-23-48.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="1011" data-original-width="1523" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZTrMaIoDRX15ztZS-oo5RIqVMHOBlGD2UCkgaCNTan091GUkDg9iaZqFjKFSfXKPolPkk6i1sWPYYnQdTMRjYxY1I2i7-_XuEklOTHF49sXOZZ9AfuSNhLcyY0hTHDYAX4nSa2z0MyFIzJw-R0glmB8kgi_sDTPv-dRmrxLIhiOInhbK7EWHzceLNC3I/s320/Screenshot%20from%202023-09-09%2014-23-48.png"/></a></div>
<p>We can verify that user is not imported, by clicking on User and View all users</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh52dXieNFJ96dX_vEunV_QD3U8D1DogrEoqGagQ7fdA0EPCzir-FZLALkMwxojJcm8hz5Slmr7Laj3c1j001lvjm0iC7cQcU6dzwDFOaXUBtSG5rsryLk5BkF6Tta7ctoyBgmZCNk6-GESHYFGQ9N5uSMDRM0NLuPw2CZF_1Dsubkv8pV79lx8DpHfqu4/s1523/Screenshot%20from%202023-09-09%2014-31-42.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="759" data-original-width="1523" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh52dXieNFJ96dX_vEunV_QD3U8D1DogrEoqGagQ7fdA0EPCzir-FZLALkMwxojJcm8hz5Slmr7Laj3c1j001lvjm0iC7cQcU6dzwDFOaXUBtSG5rsryLk5BkF6Tta7ctoyBgmZCNk6-GESHYFGQ9N5uSMDRM0NLuPw2CZF_1Dsubkv8pV79lx8DpHfqu4/s320/Screenshot%20from%202023-09-09%2014-31-42.png"/></a></div>
<p>Now test federation, by logging in to user portal: <a href="http://localhost:8180/auth/realms/demo/account/">http://localhost:8180/auth/realms/demo/account/</a></p>
<pre><code class="bash"
>john
bitnami1
kate
bitnami1
</code></pre>
<p>And observe server log, that user is fetch by RDN LDAP attribute: uid</p>
<pre><code class="bash"
>LdapOperation: search
baseDn: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(uid=john)(objectclass=inetOrgPerson)(objectclass=organizationalPerson))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
resultSize: 1
took: 2 ms
LdapOperation: search
baseDn: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(uid=john)(objectclass=inetOrgPerson)(objectclass=organizationalPerson))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
resultSize: 1
took: 1 ms
LdapOperation: search
baseDn: ou=Groups,dc=magnuskkarlsson,dc=se
filter: (&(member=cn=john,ou=People,dc=magnuskkarlsson,dc=se)(objectclass=groupOfNames))
searchScope: 1
returningAttrs: [cn]
resultSize: 2
took: 1 ms
bitnami1
</code></pre>
<h1>Kecloak Configuration User Federation with Import</h1>
<p><b>Now Import Users: ON</b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgby00-xk-Ymh9AFt-nN0hVNgRh-UFeauDWEvWOo8O6JI2WP3BE9dwVWD3Q-xE4g3Jkj8I4A09WIhJ74jzN88hK7lCYG1zi_8ZTq8Ph2DtyzU45W-_WR-pB37FiDvx5DyCoXobil-MmM9gKpjvFQ4qswaRSAbEfiph0AlqJ0QzwrRi67hR0Oz2b-jgX8RI/s2167/Screenshot%20from%202023-09-09%2015-36-21.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="400" data-original-height="2167" data-original-width="1523" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgby00-xk-Ymh9AFt-nN0hVNgRh-UFeauDWEvWOo8O6JI2WP3BE9dwVWD3Q-xE4g3Jkj8I4A09WIhJ74jzN88hK7lCYG1zi_8ZTq8Ph2DtyzU45W-_WR-pB37FiDvx5DyCoXobil-MmM9gKpjvFQ4qswaRSAbEfiph0AlqJ0QzwrRi67hR0Oz2b-jgX8RI/s400/Screenshot%20from%202023-09-09%2015-36-21.png"/></a></div>
<p>Then login (user fetced by RDN LDAP attribute: uid), close private window and login again. Now is the user fetched by UUID LDAP attribute: entryUUID</p>
<pre><code class="bash"
>LdapOperation: search
baseDn: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(uid=john)(objectclass=inetOrgPerson)(objectclass=organizationalPerson))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
resultSize: 1
took: 2 ms
LdapOperation: lookupById
baseDN: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
took: 3 ms
LdapOperation: search
baseDn: ou=Groups,dc=magnuskkarlsson,dc=se
filter: (&(member=cn=john,ou=People,dc=magnuskkarlsson,dc=se)(objectclass=groupOfNames))
searchScope: 1
returningAttrs: [cn]
resultSize: 2
took: 1 ms
...
LdapOperation: lookupById
baseDN: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
took: 4 ms
</code></pre>
<p>When User is imported you can search and find User in Web Console and you can see in server.log that User is updated from LDAP.</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEEuxzbiglxnKKo6F0DosBP5Vt2eC9mzu45l3tallisZcI3kaN8xim4aZQMYntqG14yYd2Pbds7dJqTKjCcldHzotu9FQk4tiT1IBxF7o1iEjNhgdVl9e7bSihDhp4Xhyqgk4lBbcOCstGQHO1EZG33QwTSr2rp9r35SLIhlnvDTNwwSFbOj5pOX25bnc/s1523/Screenshot%20from%202023-09-09%2014-38-44.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="785" data-original-width="1523" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEEuxzbiglxnKKo6F0DosBP5Vt2eC9mzu45l3tallisZcI3kaN8xim4aZQMYntqG14yYd2Pbds7dJqTKjCcldHzotu9FQk4tiT1IBxF7o1iEjNhgdVl9e7bSihDhp4Xhyqgk4lBbcOCstGQHO1EZG33QwTSr2rp9r35SLIhlnvDTNwwSFbOj5pOX25bnc/s320/Screenshot%20from%202023-09-09%2014-38-44.png"/></a></div>
<pre><code class="bash"
>LdapOperation: lookupById
baseDN: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
took: 2 ms
LdapOperation: lookupById
baseDN: ou=People,dc=magnuskkarlsson,dc=se
filter: (&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))
searchScope: 1
returningAttrs: [uid, userPassword, modifyTimestamp, cn, mail, createTimestamp, sn]
took: 1 ms
</code></pre>
<h1>Side effects</h1>
<p>This online dependency has also consequence, if LDAP server is offline than User cannot log in.</p>
<p>But if the LDAP server comes back online, User can login again.</p>
<pre><code class="bash"
>15:04:09,715 WARN [org.keycloak.services] (default task-44) KC-SERVICES0013: Failed authentication: org.keycloak.models.ModelException: LDAP Query failed
...
Caused by: org.keycloak.models.ModelException: Querying of LDAP failed org.keycloak.storage.ldap.idm.query.internal.LDAPQuery@54e2e27
...
Caused by: org.keycloak.models.ModelException: Could not query server using DN [ou=People,dc=magnuskkarlsson,dc=se] and filter [(&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))]
...
Caused by: javax.naming.CommunicationException: localhost:1389 [Root exception is java.net.ConnectException: Connection refused (Connection refused)]
...
Caused by: java.net.ConnectException: Connection refused (Connection refused)
...
15:04:09,755 WARN [org.keycloak.events] (default task-44) type=LOGIN_ERROR, realmId=demo, clientId=account-console, userId=null, ipAddress=127.0.0.1, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code, redirect_uri=http://localhost:8180/auth/realms/demo/account/#/, code_id=618ac086-9b7f-4bd1-b8a2-0498ab2e7390, username=john, authSessionParentId=618ac086-9b7f-4bd1-b8a2-0498ab2e7390, authSessionTabId=XLmqDMVrjkc
</code></pre>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrGXhg0w0Abq4PED5svEBAtfvy9HsRnHHJi3cajFNFg0WbDrOd4gxLjubO5nxB20INXHKrGktmBFtXSIvCW3NV3hks_rWet79GJ2AE8yYAEU31nfIp7oIH-cIqkfUUu5hGbGfspxIxeADh2aGW52bwAwA8lVdQ1sxX5BS37KETSpdUtXuOSBKR2idjx3Y/s1418/Screenshot%20from%202023-09-08%2019-16-43.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="541" data-original-width="1418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrGXhg0w0Abq4PED5svEBAtfvy9HsRnHHJi3cajFNFg0WbDrOd4gxLjubO5nxB20INXHKrGktmBFtXSIvCW3NV3hks_rWet79GJ2AE8yYAEU31nfIp7oIH-cIqkfUUu5hGbGfspxIxeADh2aGW52bwAwA8lVdQ1sxX5BS37KETSpdUtXuOSBKR2idjx3Y/s320/Screenshot%20from%202023-09-08%2019-16-43.png"/></a></div>
<p>Or if LDAP server is slow (We have set Connection and Read timeout, this is important, since default values are infinite), the User cannot login.</p>
<pre><code class="bash"
>15:10:14,894 WARN [org.keycloak.services] (default task-44) KC-SERVICES0013: Failed authentication: org.keycloak.models.ModelException: LDAP Query failed
...
Caused by: org.keycloak.models.ModelException: Querying of LDAP failed org.keycloak.storage.ldap.idm.query.internal.LDAPQuery@c684144
...
Caused by: org.keycloak.models.ModelException: Could not query server using DN [ou=People,dc=magnuskkarlsson,dc=se] and filter [(&(objectClass=*)(entryUUID=b8f499ce-e341-103d-9c19-dfc37858caec))]
...
Caused by: javax.naming.CommunicationException: localhost:1389 [Root exception is java.net.SocketTimeoutException: connect timed out]
...
Caused by: java.net.SocketTimeoutException: connect timed out
...
15:10:14,907 WARN [org.keycloak.events] (default task-44) type=LOGIN_ERROR, realmId=demo, clientId=account-console, userId=null, ipAddress=127.0.0.1, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code, redirect_uri=http://localhost:8180/auth/realms/demo/account/#/, code_id=f68f708a-82de-4c5b-957f-9c7084374922, username=john, authSessionParentId=f68f708a-82de-4c5b-957f-9c7084374922, authSessionTabId=RyZud1gnRGs
</code></pre>
<p>Another side effect of not periodically import all User, is that User, that have never logged in, will <i>not be visible in RH SSO</i>.</p>
<p>And another side effect, is that User in RH SSO will never be <b>pruned</b>, i.e. if user is deleted in LDAP, that User will never be removed from RH SSO.</p>
<p>And this online dependency, can not be <i>circumvented, by disabling user federation</i>.</p>
<h1>Kecloak Configuration User Federation and Different Edit Modes</h1>
<p><b>WRITABLE</b> means data will be synced back to LDAP on demand.</p>
<p><b>UNSYNCED</b> means user data will be imported, but not synced back to LDAP.</p>
<p><b>READ_ONLY</b> is a read-only LDAP store. "You cannot change the username, email, first name, last name, and other <b>mapped attributes</b>. Red Hat Single Sign-On shows an error anytime a user attempts to update these fields. Password updates are not supported."</p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-66197190481063942492023-09-08T12:40:00.006+02:002023-09-08T19:12:00.837+02:00OpenLDAP Image and Custom LDIF with User and Group<p>Reference: <a href="https://hub.docker.com/r/bitnami/openldap">https://hub.docker.com/r/bitnami/openldap</a></p>
<pre><code class="bash"
>LDAP_PORT_NUMBER: The port OpenLDAP is listening for requests. Priviledged port is supported (e.g. 1389). Default: 1389 (non privileged port).
LDAP_ROOT: LDAP baseDN (or suffix) of the LDAP tree. Default: dc=example,dc=org
LDAP_ADMIN_USERNAME: LDAP database admin user. Default: admin
LDAP_ADMIN_PASSWORD: LDAP database admin password. Default: adminpassword
LDAP_CONFIG_ADMIN_ENABLED: Whether to create a configuration admin user. Default: no.
LDAP_USERS: Comma separated list of LDAP users to create in the default LDAP tree. Default: user01,user02
LDAP_PASSWORDS: Comma separated list of passwords to use for LDAP users. Default: bitnami1,bitnami2
LDAP_USER_DC: DC for the users' organizational unit. Default: users
LDAP_GROUP: Group used to group created users. Default: readers
LDAP_ALLOW_ANON_BINDING: Allow anonymous bindings to the LDAP server. Default: yes.
LDAP_PASSWORD_HASH: Hash to be used in generation of user passwords. Must be one of {SSHA}, {SHA}, {SMD5}, {MD5}, {CRYPT}, and {CLEARTEXT}. Default: {SSHA}.
LDAP_CUSTOM_LDIF_DIR: Location of a directory that contains LDIF files that should be used to bootstrap the database. Only files ending in .ldif will be used. Default LDAP tree based on the LDAP_USERS, LDAP_PASSWORDS, LDAP_USER_DC and LDAP_GROUP will be skipped when LDAP_CUSTOM_LDIF_DIR is used. When using this it will override the usage of LDAP_USERS, LDAP_PASSWORDS, LDAP_USER_DC and LDAP_GROUP. You should set LDAP_ROOT to your base to make sure the olcSuffix configured on the database matches the contents imported from the LDIF files. Default: /ldifs
LDAP_PASSWORD_HASH: Hash to be used in generation of user passwords. Must be one of {SSHA}, {SHA}, {SMD5}, {MD5}, {CRYPT}, and {CLEARTEXT}. Default: {SSHA}.
</code></pre>
<p>Create a new directory ldif with custom LDIF for Users and Groups</p>
<pre><code class="bash"
>dn: dc=magnuskkarlsson,dc=se
objectClass: dcObject
objectClass: organization
dc: magnuskkarlsson
o: Magnus K Karlsson
dn: ou=People,dc=magnuskkarlsson,dc=se
objectClass: organizationalUnit
ou: People
dn: ou=Groups,dc=magnuskkarlsson,dc=se
objectClass: organizationalUnit
ou: Groups
## Users
dn: cn=john,ou=People,dc=magnuskkarlsson,dc=se
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: john
userPassword:: Yml0bmFtaTE=
cn: John
sn: Doe
mail: john.doe@domain.com
uidNumber: 1000
gidNumber: 1000
homeDirectory: /home/john
dn: cn=kate,ou=People,dc=magnuskkarlsson,dc=se
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: kate
userPassword:: Yml0bmFtaTE=
cn: Kate
sn: Doe
mail: kate.doe@domain.com
uidNumber: 1001
gidNumber: 1001
homeDirectory: /home/kate
## Groups
dn: cn=USER,ou=Groups,dc=magnuskkarlsson,dc=se
cn: USER
objectClass: groupOfNames
member: cn=john,ou=People,dc=magnuskkarlsson,dc=se
member: cn=kate,ou=People,dc=magnuskkarlsson,dc=se
dn: cn=ADMIN,ou=Groups,dc=magnuskkarlsson,dc=se
cn: ADMIN
objectClass: groupOfNames
member: cn=john,ou=People,dc=magnuskkarlsson,dc=se
</code></pre>
<pre><code class="bash"
>$ podman run -d --name openldap \
-e LDAP_ROOT=dc=magnuskkarlsson,dc=se \
-e LDAP_ADMIN_USERNAME=admin \
-e LDAP_ADMIN_PASSWORD=changeit \
-e LDAP_CONFIG_ADMIN_ENABLED=true \
-e LDAP_ALLOW_ANON_BINDING=false \
-e LDAP_CUSTOM_LDIF_DIR=/ldifs \
-p 1389:1389 \
-p 1636:1636 \
-v ./ldifs:/ldifs:Z \
docker.io/bitnami/openldap:2.6
</code></pre>
<pre><code class="bash"
>$ podman logs --follow openldap
...
13:30:55.23 INFO ==> Loading custom LDIF files...
13:30:55.23 WARN ==> Ignoring LDAP_USERS, LDAP_PASSWORDS, LDAP_USER_DC and LDAP_GROUP environment variables...
13:30:56.35 INFO ==> ** LDAP setup finished! **
</code></pre>
<p>And later to stop</p>
<pre><code class="bash"
>$ podman stop openldap; podman rm openldap
</code></pre>
<p>Now verify ldap and it's entries. First install ldap client</p>
<pre><code class="bash"
>$ sudo dnf install openldap-clients
</code></pre>
<pre><code class="bash"
>$ ldapsearch -h
...
-H URI LDAP Uniform Resource Identifier(s)
-D binddn bind DN
-x Simple authentication
-w passwd bind password (for simple authentication)
-b basedn base dn for search
-s scope one of base, one, sub or children (search scope)
...
$ ldapsearch -H ldap://localhost:1389 -D cn=admin,dc=magnuskkarlsson,dc=se -w changeit -b dc=magnuskkarlsson,dc=se -s sub
</code></pre>
<p>GUI Administration Tools. Apache Directory Studio Eclipse-based LDAP tools</p>
<p><a href="https://magnus-k-karlsson.blogspot.com/2015/02/understanding-ldap-and-ldap.html">https://magnus-k-karlsson.blogspot.com/2015/02/understanding-ldap-and-ldap.html</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-47349871561194991882023-08-30T12:46:00.001+02:002023-08-30T12:46:12.346+02:00 Spring Boot 3 with X509 Authentication and JDBC Storage <h1>Reference</h1>
<ul>
<li><a href="https://docs.spring.io/spring-security/reference/servlet/authentication/x509.html">https://docs.spring.io/spring-security/reference/servlet/authentication/x509.html</a></li>
<li><a href="https://magnus-k-karlsson.blogspot.com/2023/08/using-java-17-keytool-as-root-ca-to.html">https://magnus-k-karlsson.blogspot.com/2023/08/using-java-17-keytool-as-root-ca-to.html</a></li>
<li><a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-with-basic-authentication.html">https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-with-basic-authentication.html</a></li>
</ul>
<h1>Maven</h1>
<p>Using same pom, but different artifactId as previous blog: <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-with-basic-authentication.html">Basic Auth and JDBC Password Storage</a></p>
<h1>MySQL</h1>
<p>Using same docker mysql container wiht same data as in previous blog: <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-with-basic-authentication.html">Basic Auth and JDBC Password Storage</a></p>
<h1>Server and User Certificate and Truststore</h1>
<p>Using same root ca, certs and truststore as in previous blog: <a href="https://magnus-k-karlsson.blogspot.com/2023/08/using-java-17-keytool-as-root-ca-to.html">Using Java 17 keytool as Root CA to create Server and User Certificate</a></p>
<h1>Applicaiton</h1>
<h2>src/main/resources/application.properties</h2>
<pre><code class="java"
>server.port = 8443
# https://docs.spring.io/spring-security/reference/servlet/architecture.html#servlet-logging
logging.level.org.springframework.security = TRACE
# https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
#server.ssl.bundle
#server.ssl.certificate
#server.ssl.certificate-private-key
#server.ssl.ciphers
server.ssl.client-auth = want
server.ssl.enabled = true
#server.ssl.enabled-protocols
server.ssl.key-alias = localhost
server.ssl.key-password = changeit
server.ssl.key-store = localhost.p12
server.ssl.key-store-password = changeit
#server.ssl.key-store-provider
server.ssl.key-store-type = PKCS12
#server.ssl.protocol
#server.ssl.trust-certificate
#server.ssl.trust-certificate-private-key =
server.ssl.trust-store = truststore.jks
server.ssl.trust-store-password = changeit
#server.ssl.trust-store-provider
server.ssl.trust-store-type = JKS
#server.servlet.session.cookie.domain
server.servlet.session.cookie.http-only = true
server.servlet.session.cookie.max-age = 3m
#server.servlet.session.cookie.name
#server.servlet.session.cookie.path
server.servlet.session.cookie.same-site = strict
server.servlet.session.cookie.secure = true
server.servlet.session.persistent = false
#server.servlet.session.store-dir
server.servlet.session.timeout = 3m
server.servlet.session.tracking-modes = cookie
spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=user
spring.datasource.password=changeit
</code></pre>
<p>Spring Java configuration</p>
<pre><code class="java"
>package se.mkk.springboot3x509jdbc;
import javax.sql.DataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.provisioning.JdbcUserDetailsManager;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.logout.HeaderWriterLogoutHandler;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter.Directive;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class X509JdbcSecurityConfig {
private final Logger log = LoggerFactory.getLogger(this.getClass());
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.sessionManagement(
// https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html
// https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
Customizer.withDefaults())
.authorizeHttpRequests(authorizeHttpRequests -> authorizeHttpRequests //
.requestMatchers("/login", "/logout").permitAll() //
.anyRequest().authenticated()) //
.x509(x509 -> x509 //
// https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/basic.html
.subjectPrincipalRegex("CN=(.*?),") //
.userDetailsService(this.userDetailsService())) //
.csrf(csrf -> csrf //
// https://docs.spring.io/spring-security/reference/servlet/exploits/csrf.html#csrf-token-repository-cookie
.ignoringRequestMatchers("/logout", "/api"))
.logout(logout -> logout //
// https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html#clear-all-site-data
.addLogoutHandler(new HeaderWriterLogoutHandler(new ClearSiteDataHeaderWriter(Directive.ALL))));
return http.build();
}
// private UserDetailsService userDetailsService() {
// return username -> {
// log.info("X509 {}", username);
// return User.withUsername(username).password("DUMMY").roles("USER", "ADMIN").build();
// };
// }
// https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/jdbc.html
@Autowired
private DataSource dataSource;
private UserDetailsService userDetailsService() {
JdbcUserDetailsManager users = new JdbcUserDetailsManager(dataSource);
return users;
}
}
</code></pre>
<p>Test REST endpoint</p>
<pre><code class="java"
>package se.mkk.springboot3x509jdbc;
import java.security.Principal;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.security.core.Authentication;
import org.springframework.web.bind.annotation.CrossOrigin;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpSession;
@RestController
@RequestMapping("/api/users")
@CrossOrigin(origins = "http://localhost:4200")
public class UserController {
@GetMapping
public Map<String, String> getUser(HttpServletRequest request, HttpSession session, Principal principal) {
Map<String, String> rtn = new LinkedHashMap<>();
rtn.put("session_getMaxInactiveInterval_sec", session.getMaxInactiveInterval() + "s");
rtn.put("session_getLastAccessedTime",
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss Z").format(new Date(session.getLastAccessedTime())));
rtn.put("request_getRemoteUser", request.getRemoteUser());
rtn.put("request_isUserInRole_USER", Boolean.toString(request.isUserInRole("USER")));
rtn.put("request_getUserPrincipal_getClass", request.getUserPrincipal().getClass().getName());
rtn.put("principal_getClass_getName", principal.getClass().getName());
rtn.put("principal_getName", principal.getName());
if (principal instanceof Authentication authentication) {
List<String> authorities = authentication.getAuthorities().stream()
.map(grantedAuthority -> grantedAuthority.getAuthority()).toList();
rtn.put("JwtAuthenticationToken.getAuthorities()", authorities.toString());
}
return rtn;
}
}
</code></pre>
<h1>Test</h1>
<p>We need to convert p12 to pem files for curl.</p>
<pre><code class="bash"
>$ openssl pkcs12 -in john.p12 -out john.pem -nodes
</code></pre>
<pre><code class="bash"
>$ curl -v -X GET --cacert RootCA.cert.pem --cert john.cert.pem --key john.key.pem https://localhost:8443/api/users
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:8443...
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: RootCA.cert.pem
* CApath: none
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
* subject: C=SE; O=Server; OU=Localhost_Server; CN=localhost
* start date: Aug 30 09:35:00 2023 GMT
* expire date: Nov 28 09:35:00 2023 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: C=SE; O=CertificateAuthority; OU=Root_CertificateAuthority; CN=RootCA
* SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /api/users HTTP/1.1
> Host: localhost:8443
> User-Agent: curl/7.85.0
> Accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Set-Cookie: JSESSIONID=5358CF4B62B84A8E99C4DB183CB2BDD2; Max-Age=180; Expires=Wed, 30 Aug 2023 10:07:14 GMT; Path=/; Secure; HttpOnly; SameSite=Strict
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 0
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Strict-Transport-Security: max-age=31536000 ; includeSubDomains
< X-Frame-Options: DENY
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Wed, 30 Aug 2023 10:04:14 GMT
<
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host localhost left intact
{"session_getMaxInactiveInterval_sec":"180s","session_getLastAccessedTime":"2023-08-30 12:04:14 +0200","request_getRemoteUser":"john","request_isUserInRole_USER":"true","request_getUserPrincipal_getClass":"org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationToken","principal_getClass_getName":"org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationToken","principal_getName":"john","JwtAuthenticationToken.getAuthorities()":"[ROLE_ADMIN, ROLE_USER]"}
</code></pre>
<pre><code class="bash"
>$ curl -s -X GET --cacert RootCA.cert.pem --cert john.cert.pem --key john.key.pem https://localhost:8443/api/users | jq -r
{
"session_getMaxInactiveInterval_sec": "180s",
"session_getLastAccessedTime": "2023-08-30 12:04:44 +0200",
"request_getRemoteUser": "john",
"request_isUserInRole_USER": "true",
"request_getUserPrincipal_getClass": "org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationToken",
"principal_getClass_getName": "org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationToken",
"principal_getName": "john",
"JwtAuthenticationToken.getAuthorities()": "[ROLE_ADMIN, ROLE_USER]"
}
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-22027918146454599062023-08-30T12:32:00.005+02:002023-08-30T12:33:46.370+02:00Using Java 17 keytool as Root CA to create Server and User Certificate<h1>Reference</h1>
<ul>
<li><a href="https://docs.oracle.com/en/java/javase/17/docs/specs/man/keytool.html">https://docs.oracle.com/en/java/javase/17/docs/specs/man/keytool.html</a></li>
<li><a href="https://magnus-k-karlsson.blogspot.com/2020/02/x509-certificate-profiles.html">https://magnus-k-karlsson.blogspot.com/2020/02/x509-certificate-profiles.html</a></li>
</ul>
<h1>Root CA</h1>
<p>Generate Self Signed Root CA</p>
<pre><code class="bash"
>$ keytool -genkeypair -alias RootCA -dname "cn=RootCA, ou=Root_CertificateAuthority, o=CertificateAuthority, c=SE" -validity 3650 -keyalg RSA -keysize 4096 -ext bc:c -keystore RootCA.p12 -storetype PKCS12 -storepass changeit -keypass changeit
</code></pre>
<p>Export Root CA certificate and truststore with Root CA</p>
<pre><code class="bash"
>$ keytool -exportcert -alias RootCA -keystore RootCA.p12 -storetype PKCS12 -storepass changeit -rfc -file RootCA.cert.pem
$ keytool -importcert -alias RootCA -trustcacerts -noprompt -keystore truststore.jks -storetype JKS -storepass changeit -keypass changeit -file RootCA.cert.pem
</code></pre>
<h1>Server certificate</h1>
<p>Generate Server certificate keypair.</p>
<pre><code class="bash"
>$ keytool -genkeypair -alias localhost -dname "cn=localhost, ou=Localhost_Server, o=Server, c=SE" -validity 730 -keyalg RSA -keysize 2048 -keystore localhost.p12 -storetype PKCS12 -storepass changeit -keypass changeit
</code></pre>
<p>Generate CSR and sign with Root CA</p>
<pre><code class="bash"
>$ keytool -certreq -alias localhost -keystore localhost.p12 -storetype PKCS12 -storepass changeit | \
keytool -gencert -alias RootCA -keystore RootCA.p12 -storetype PKCS12 -storepass changeit -ext ku:c=dig,keyEnc -ext "san=dns:localhost,ip:127.0.0.1" -ext eku=serverAuth -rfc > localhost.cert.pem
</code></pre>
<p>Create certificate chain</p>
<pre><code class="bash"
>$ cat RootCA.cert.pem >> localhost.cert.pem
</code></pre>
<p>Import/replace self signed certificate with Root signed</p>
<pre><code class="bash"
>$ keytool -importcert -alias localhost -trustcacerts -noprompt -keystore localhost.p12 -storetype PKCS12 -storepass changeit -file localhost.cert.pem
</code></pre>
<p>Print and verify </p>
<pre><code class="bash"
>$ keytool -list -keystore localhost.p12 -storepass changeit -v
</code></pre>
<h1>User Certificate</h1>
<p>Generate User certificate keypair</p>
<pre><code class="bash"
>$ keytool -genkeypair -alias john -dname "cn=john, ou=John_User, o=User, c=SE" -validity 730 -keyalg RSA -keysize 2048 -keystore john.p12 -storetype PKCS12 -storepass changeit -keypass changeit
</code></pre>
<p>Generate CSR and sign with Root CA</p>
<pre><code class="bash"
>$ keytool -certreq -alias john -keystore john.p12 -storetype PKCS12 -storepass changeit | \
keytool -gencert -alias RootCA -keystore RootCA.p12 -storetype PKCS12 -storepass changeit -ext ku:c=digitalSignature,nonRepudiation,keyEncipherment -ext eku=clientAuth,emailProtection -rfc > john.cert.pem
</code></pre>
<p>Create certificate chain</p>
<pre><code class="bash"
>$ cat RootCA.cert.pem >> john.cert.pem
</code></pre>
<p>Import/replace self signed certificate with Root signed</p>
<pre><code class="bash"
>$ keytool -importcert -alias john -trustcacerts -noprompt -keystore john.p12 -storetype PKCS12 -storepass changeit -file john.cert.pem
</code></pre>
<p>Print and verify </p>
<pre><code class="bash"
>$ keytool -list -keystore john.p12 -storepass changeit -v
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-8564761286068683892023-08-29T23:14:00.000+02:002023-08-29T23:14:18.693+02:00Spring Boot 3 with Basic Authentication and JDBC Password Storage<h3>Create new Maven Project</h3>
<p>Create new project with Spring Initializr (https://start.spring.io/) and add dependency: Spring Data JDBC and MySQL</p>
<pre><code class="bash"
> <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
</dependency>
</code></pre>
<h3>Add Spring Data JDBC properties <b>src/main/resources/application.properties</b></h3>
<p><a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.data">https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.data</a></p>
<pre><code class="java"
> spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=user
spring.datasource.password=changeit
</code></pre>
<h3>Start a local mysql server with a container image.</h3>
<p><a href="https://catalog.redhat.com/software/containers/rhel9/mysql-80/61a60915c17162a20c1c6a34">https://catalog.redhat.com/software/containers/rhel9/mysql-80/61a60915c17162a20c1c6a34</a></p>
<p><a href="https://hub.docker.com/_/mysql">https://hub.docker.com/_/mysql</a></p>
<pre><code class="bash"
>$ podman run -d --name mysqld \
-e MYSQL_USER=user \
-e MYSQL_PASSWORD=changeit \
-e MYSQL_DATABASE=mydb \
-e MYSQL_ROOT_PASSWORD=changeit \
-p 3306:3306 \
docker.io/library/mysql:8.0
$ podman logs --follow mysqld
</code></pre>
<p>Create and populate DB. Use below test method to generated hashed password.</p>
<pre><code class="java"
> @Test
public void test() throws Exception {
PasswordEncoder passwordEncoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
System.out.println(passwordEncoder.encode("changeit"));
}
</code></pre>
<p>Now connect to mysql container interactive and execute sql commands to creae tables and populate them with data.</p>
<p><a href="https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/jdbc.html">https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/jdbc.html</a></p>
<pre><code class="bash"
>$ podman exec -it mysqld /bin/bash
bash-4.4# mysql -u root -p mydb
create table users(username varchar(50) not null primary key, password varchar(500) not null, enabled boolean not null);
create table authorities (username varchar(50) not null, authority varchar(50) not null, constraint fk_authorities_users foreign key(username) references users(username));
create unique index ix_auth_username on authorities (username, authority);
INSERT INTO users (username, password, enabled) VALUES ('john', '{bcrypt}$2a$10$aohc8ylx1YcZx6p/L2BRv.I4oQfDin9Ed2CNTy0ZXQ3ZpdiMalLp6', true);
INSERT INTO authorities (username, authority) VALUES ('john', 'ROLE_USER');
INSERT INTO authorities (username, authority) VALUES ('john', 'ROLE_ADMIN');
</code></pre>
<h3>Application</h3>
<pre><code class="bash"
>package se.mkk.springboot3basicjdbc;
import javax.sql.DataSource;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.provisioning.JdbcUserDetailsManager;
import org.springframework.security.provisioning.UserDetailsManager;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.logout.HeaderWriterLogoutHandler;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter.Directive;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class BasicJdbcSecurityConfig {
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.sessionManagement(
// https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html
// https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
Customizer.withDefaults())
.authorizeHttpRequests(authorizeHttpRequests -> authorizeHttpRequests //
.requestMatchers("/login", "/logout").permitAll() //
.anyRequest().authenticated()) //
.httpBasic( //
// https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/basic.html
Customizer.withDefaults()) //
.csrf(csrf -> csrf //
// https://docs.spring.io/spring-security/reference/servlet/exploits/csrf.html#csrf-token-repository-cookie
.ignoringRequestMatchers("/logout", "/api"))
.logout(logout -> logout //
// https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html#clear-all-site-data
.addLogoutHandler(new HeaderWriterLogoutHandler(new ClearSiteDataHeaderWriter(Directive.ALL))));
return http.build();
}
// https://docs.spring.io/spring-security/reference/servlet/authentication/passwords/jdbc.html
@Bean
public UserDetailsManager jdbcUserDetailsManager(DataSource dataSource) {
JdbcUserDetailsManager users = new JdbcUserDetailsManager(dataSource);
return users;
}
}
</code></pre>
<pre><code class="bash"
>package se.mkk.springboot3basicjdbc;
import java.security.Principal;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.security.core.Authentication;
import org.springframework.web.bind.annotation.CrossOrigin;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpSession;
@RestController
@RequestMapping("/api/users")
@CrossOrigin(origins = "http://localhost:4200")
public class UserController {
@GetMapping
public Map<String, String> getUser(HttpServletRequest request, HttpSession session, Principal principal) {
Map<String, String> rtn = new LinkedHashMap<>();
rtn.put("session_getMaxInactiveInterval_sec", session.getMaxInactiveInterval() + "s");
rtn.put("session_getLastAccessedTime",
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss Z").format(new Date(session.getLastAccessedTime())));
rtn.put("request_getRemoteUser", request.getRemoteUser());
rtn.put("request_isUserInRole_USER", Boolean.toString(request.isUserInRole("USER")));
rtn.put("request_getUserPrincipal_getClass", request.getUserPrincipal().getClass().getName());
rtn.put("principal_getClass_getName", principal.getClass().getName());
rtn.put("principal_getName", principal.getName());
if (principal instanceof Authentication authentication) {
List<String> authorities = authentication.getAuthorities().stream()
.map(grantedAuthority -> grantedAuthority.getAuthority()).toList();
rtn.put("JwtAuthenticationToken.getAuthorities()", authorities.toString());
}
return rtn;
}
}
</code></pre>
<p>Test</p>
<pre><code class="bash"
>$ curl -v -X GET -u "john:changeit" http://localhost:8080/api/users
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'john'
> GET /api/users HTTP/1.1
> Host: localhost:8080
> Authorization: Basic am9objpjaGFuZ2VpdA==
> User-Agent: curl/7.85.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Set-Cookie: JSESSIONID=56FAD7EB738EB12B558F02EBC04122BE; Max-Age=180; Expires=Tue, 29 Aug 2023 21:02:41 GMT; Path=/; Secure; HttpOnly; SameSite=Strict
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 0
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< X-Frame-Options: DENY
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Tue, 29 Aug 2023 20:59:41 GMT
<
* Connection #0 to host localhost left intact
{"session_getMaxInactiveInterval_sec":"180s","session_getLastAccessedTime":"2023-08-29 22:59:41 +0200","request_getRemoteUser":"john","request_isUserInRole_USER":"true","request_getUserPrincipal_getClass":"org.springframework.security.authentication.UsernamePasswordAuthenticationToken","principal_getClass_getName":"org.springframework.security.authentication.UsernamePasswordAuthenticationToken","principal_getName":"john","JwtAuthenticationToken.getAuthorities()":"[ROLE_ADMIN, ROLE_USER]"}
$ curl -s -X GET -u "john:changeit" http://localhost:8080/api/users | jq -r
{
"session_getMaxInactiveInterval_sec": "180s",
"session_getLastAccessedTime": "2023-08-29 22:58:32 +0200",
"request_getRemoteUser": "john",
"request_isUserInRole_USER": "true",
"request_getUserPrincipal_getClass": "org.springframework.security.authentication.UsernamePasswordAuthenticationToken",
"principal_getClass_getName": "org.springframework.security.authentication.UsernamePasswordAuthenticationToken",
"principal_getName": "john",
"JwtAuthenticationToken.getAuthorities()": "[ROLE_ADMIN, ROLE_USER]"
}
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-57161392982561739812023-08-23T14:25:00.004+02:002023-08-23T14:25:32.700+02:00Oauth 2.0 Client Credentials Grant<p>Used for machine-to-machine communication</p>
<p>Client needs to hold secrets, since Access Token is directly exposed</p>
<p><a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.4.2">https://datatracker.ietf.org/doc/html/rfc6749#section-4.4.2</a></p>
<pre><code class="bash"
> POST /token HTTP/1.1
Host: server.example.com
Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
grant_type
REQUIRED. Value MUST be set to "client_credentials".
scope
OPTIONAL. The scope of the access request as described by
Section 3.3.
</code></pre>
<pre><code class="bash"
>$ curl -s -X POST \
-H 'Content-Type: application/x-www-form-urlencoded' \
-u 'spring-boot3-oauth2-login:jO09Uwhi8oxTL3QnTKtYZ20ByQvB2qA0' \
http://localhost:8180/auth/realms/demo/protocol/openid-connect/token \
-d "grant_type=client_credentials&scope=openid&client_id=spring-boot3-oauth2-login" | jq -r
{
"access_token": "eyJhbGci...tYfo6rEPEakNg",
"expires_in": 180,
"refresh_expires_in": 0,
"token_type": "Bearer",
"id_token": "eyJhbGci...KaoQ",
"not-before-policy": 0,
"scope": "openid email profile"
}
</code></pre>
<p>Parsed Access Token</p>
<pre><code class="json"
>{
"exp": 1692791418,
"iat": 1692791238,
"jti": "a30d999e-2b26-4720-92a1-d907161675a0",
"iss": "http://localhost:8180/auth/realms/demo",
"aud": "account",
"sub": "0fb7c670-ae44-412c-9da1-cf150eb0c327",
"typ": "Bearer",
"azp": "spring-boot3-oauth2-login",
"acr": "1",
"allowed-origins": [
"http://localhost:8080"
],
"realm_access": {
"roles": [
"offline_access",
"default-roles-demo",
"uma_authorization"
]
},
"resource_access": {
"account": {
"roles": [
"manage-account",
"manage-account-links",
"view-profile"
]
}
},
"scope": "openid email profile",
"email_verified": false,
"clientId": "spring-boot3-oauth2-login",
"clientHost": "127.0.0.1",
"preferred_username": "service-account-spring-boot3-oauth2-login",
"clientAddress": "127.0.0.1"
}
</code></pre>
<p>Compare with OAuth 2.0 Resource Owner Password Credentials Grant where the Access Token is request for a logged in user</p>
<pre><code class="bash"
>curl -s -X POST \
-H 'Content-Type: application/x-www-form-urlencoded' \
-u 'spring-boot3-oauth2-login:jO09Uwhi8oxTL3QnTKtYZ20ByQvB2qA0' \
http://localhost:8180/auth/realms/demo/protocol/openid-connect/token \
-d "grant_type=password&username=john&password=changeit" | jq -r
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-22592499429415370572023-08-23T11:33:00.003+02:002023-08-23T11:33:27.484+02:00Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant) and mTLS Access Token Request<h1>Introduction</h1>
<p>Using BASIC authentication when requesting Access Token in OAUth 2 Authorization Code Flow is not the safest way. Especially if this password is rarely changed.</p>
<p>See <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-spring-security-6.html">Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant)</a></p>
<p>Reference: <a href="https://docs.spring.io/spring-security/reference/servlet/oauth2/client/authorization-grants.html#_requesting_an_access_token">Spring Security OAuth2 Authorization Code Requesting an Access Token</a></p>
<h1>Prerequisite</h1>
<ul>
<li>Java 17</li>
<li>Maven 3.6.3 or later</li>
<li>Spring 3.1.2</li>
<li>Spring Security 6.1.2</li>
<li>Keycloak. I will use the commercial version RH SSO 7.6.0</li>
<li>OAuth2 Authorization Code Grand <a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.1">https://datatracker.ietf.org/doc/html/rfc6749#section-4.1</a></li>
</ul>
<h1>Application</h1>
<h2>pom.xml</h2>
<pre><code class="java"
><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.2</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>se.mkk</groupId>
<artifactId>spring-boot-3-oauth2-login-keycloak-mtls</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-boot-3-oauth2-login-keycloak-mtls</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<!-- org.springframework.http.client.HttpComponentsClientHttpRequestFactory -->
<dependency>
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
</dependency>
<!-- dev tools -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<!-- test support -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
</code></pre>
<h2>src/main/resources/application.properties</h2>
<p>To make mTLS work as authenticated when requesting Access Token, we need to send client_id in body, as is done when having public client</p>
<p>"client_id - REQUIRED, if the client is not authenticating with the authorization server"<a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.3">https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.3</a></p>
<pre><code class="java"
># OAuth2 Log In Spring Boot 2.x Property Mappings
# https://docs.spring.io/spring-security/reference/servlet/oauth2/login/core.html#oauth2login-boot-property-mappings
spring.security.oauth2.client.registration.keycloak.client-id = spring-boot3-oauth2-login
#spring.security.oauth2.client.registration.keycloak.client-secret = CHANGEME!!!
spring.security.oauth2.client.registration.keycloak.client-authentication-method = none
# org.springframework.security.oauth2.core.AuthorizationGrantType
spring.security.oauth2.client.registration.keycloak.authorization-grant-type = authorization_code
#spring.security.oauth2.client.registration.keycloak.redirect-uri =
spring.security.oauth2.client.registration.keycloak.scope = openid
#spring.security.oauth2.client.registration.keycloak.client-name =
#spring.security.oauth2.client.provider.keycloak.authorization-uri
#spring.security.oauth2.client.provider.keycloak.token-uri
#spring.security.oauth2.client.provider.keycloak.jwk-set-uri
spring.security.oauth2.client.provider.keycloak.issuer-uri = https://localhost:8543/auth/realms/demo
#spring.security.oauth2.client.provider.keycloak.user-info-uri
#spring.security.oauth2.client.provider.keycloak.user-info-authentication-method
spring.security.oauth2.client.provider.keycloak.user-name-attribute = preferred_username
</code></pre>
<h2>Spring Security Java Config</h2>
<pre><code class="java"
>package se.mkk.springboot3oauth2loginkeycloak;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.Map;
import javax.net.ssl.SSLContext;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManagerBuilder;
import org.apache.hc.client5.http.io.HttpClientConnectionManager;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactoryBuilder;
import org.apache.hc.core5.ssl.SSLContextBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.http.converter.FormHttpMessageConverter;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.client.endpoint.DefaultAuthorizationCodeTokenResponseClient;
import org.springframework.security.oauth2.client.endpoint.OAuth2AccessTokenResponseClient;
import org.springframework.security.oauth2.client.endpoint.OAuth2AuthorizationCodeGrantRequest;
import org.springframework.security.oauth2.client.http.OAuth2ErrorResponseErrorHandler;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserRequest;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserService;
import org.springframework.security.oauth2.core.OAuth2AccessToken;
import org.springframework.security.oauth2.core.http.converter.OAuth2AccessTokenResponseHttpMessageConverter;
import org.springframework.security.oauth2.core.oidc.user.DefaultOidcUser;
import org.springframework.security.oauth2.core.oidc.user.OidcUser;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.web.client.RestTemplate;
import com.nimbusds.jose.util.JSONObjectUtils;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class OAuth2LoginSecurityConfig {
@Value("${javax.net.ssl.keyStore}")
private String keyStore;
@Value("${javax.net.ssl.keyStorePassword}")
private String keyStorePassword;
@Value("${javax.net.ssl.keyStoreType:PKCS12}")
private String keyStoreType;
@Value("${javax.net.ssl.trustStore}")
private String trustStore;
@Value("${javax.net.ssl.trustStorePassword}")
private String trustStorePassword;
@Value("${javax.net.ssl.trustStoreType:jks}")
private String trustStoreType;
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/core.html#oauth2login-provide-securityfilterchain-bean
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.authorizeHttpRequests(authorize -> authorize //
.anyRequest().authenticated()) //
.oauth2Login(oauth2 -> oauth2 //
.userInfoEndpoint(userInfo -> userInfo //
.oidcUserService(this.oidcUserService())))
.oauth2Client(oauth2Client -> oauth2Client //
.authorizationCodeGrant(authorizationCodeGrant -> authorizationCodeGrant //
.accessTokenResponseClient(this.accessTokenResponseClient())));
return http.build();
}
private OAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest> accessTokenResponseClient() {
DefaultAuthorizationCodeTokenResponseClient accessTokenResponseClient = new DefaultAuthorizationCodeTokenResponseClient();
// org.springframework.security.oauth2.client.endpoint.DefaultAuthorizationCodeTokenResponseClient
RestTemplate restTemplate = new RestTemplate(
Arrays.asList(new FormHttpMessageConverter(), new OAuth2AccessTokenResponseHttpMessageConverter()));
restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
try {
// https://github.com/apache/httpcomponents-client/blob/5.2.x/httpclient5/src/test/java/org/apache/hc/client5/http/examples/ClientCustomSSL.java
SSLContext sslContext = new SSLContextBuilder() //
.loadKeyMaterial(Path.of(keyStore), keyStorePassword.toCharArray(), keyStorePassword.toCharArray()) //
.loadTrustMaterial(Path.of(trustStore), trustStorePassword.toCharArray()) //
.build();
final SSLConnectionSocketFactory sslSocketFactory = SSLConnectionSocketFactoryBuilder.create() //
.setSslContext(sslContext) //
.build();
final HttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder.create() //
.setSSLSocketFactory(sslSocketFactory) //
.build();
CloseableHttpClient httpClient = HttpClients.custom() //
.setConnectionManager(cm) //
.build();
// "uses https://hc.apache.org/httpcomponents-client-ga Apache HttpComponents HttpClient to create requests"
ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
restTemplate.setRequestFactory(requestFactory);
} catch (Exception e) {
e.printStackTrace();
}
accessTokenResponseClient.setRestOperations(restTemplate);
return accessTokenResponseClient;
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-map-authorities-oauth2userservice
private OAuth2UserService<OidcUserRequest, OidcUser> oidcUserService() {
final OidcUserService delegate = new OidcUserService();
return (userRequest) -> {
// Delegate to the default implementation for loading a user
OidcUser oidcUser = delegate.loadUser(userRequest);
OAuth2AccessToken accessToken = userRequest.getAccessToken();
Collection<GrantedAuthority> mappedAuthorities = new HashSet<>();
// 1) Fetch the authority information from the protected resource using accessToken
// 2) Map the authority information to one or more GrantedAuthority's and add it to mappedAuthorities
try {
String[] chunks = accessToken.getTokenValue().split("\\.");
Base64.Decoder decoder = Base64.getUrlDecoder();
String header = new String(decoder.decode(chunks[0]));
String payload = new String(decoder.decode(chunks[1]));
Map<String, Object> claims = JSONObjectUtils.parse(payload);
mappedAuthorities = new KeycloakAuthoritiesConverter().convert(claims);
} catch (Exception e) {
e.printStackTrace();
}
// 3) Create a copy of oidcUser but use the mappedAuthorities instead
oidcUser = new DefaultOidcUser(mappedAuthorities, oidcUser.getIdToken(), oidcUser.getUserInfo(),
"preferred_username");
return oidcUser;
};
}
// Spring OAuth2 uses default Scopes Not Roles for Authorization
// org.springframework.security.oauth2.server.resource.authentication.JwtGrantedAuthoritiesConverter
private class KeycloakAuthoritiesConverter implements Converter<Jwt, Collection<GrantedAuthority>> {
@Override
public Collection<GrantedAuthority> convert(Jwt jwt) {
return convert(jwt.getClaims());
}
public Collection<GrantedAuthority> convert(Map<String, Object> claims) {
Collection<GrantedAuthority> grantedAuthorities = new ArrayList<>();
for (String authority : getAuthorities(claims)) {
grantedAuthorities.add(new SimpleGrantedAuthority("ROLE_" + authority));
}
return grantedAuthorities;
}
private Collection<String> getAuthorities(Map<String, Object> claims) {
Object realm_access = claims.get("realm_access");
if (realm_access instanceof Map) {
Map<String, Object> map = castAuthoritiesToMap(realm_access);
Object roles = map.get("roles");
if (roles instanceof Collection) {
return castAuthoritiesToCollection(roles);
}
}
return Collections.emptyList();
}
@SuppressWarnings("unchecked")
private Map<String, Object> castAuthoritiesToMap(Object authorities) {
return (Map<String, Object>) authorities;
}
@SuppressWarnings("unchecked")
private Collection<String> castAuthoritiesToCollection(Object authorities) {
return (Collection<String>) authorities;
}
}
}
</code></pre>
<h2>Run</h2>
<p>When running the app you need to set the above Java System Properties or better set keystore properties in Spring application.properties file, so you can not list keystore password when listing processes and it's arguments from OS.</p>
<pre><code class="java"
>-Djavax.net.ssl.keyStore=client.p12 \
-Djavax.net.ssl.keyStorePassword=CHANGEME!!! \
-Djavax.net.ssl.keyStoreType=PKCS12 \
-Djavax.net.ssl.trustStore=truststore.jks \
-Djavax.net.ssl.trustStorePassword=changeit \
-Djavax.net.ssl.trustStoreType=JKS
</code></pre>
<h1>Source Code</h1>
<p><a href="https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak-mtls">https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak-mtls</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-76333949038314838622023-08-23T10:36:00.003+02:002023-08-24T11:57:57.142+02:00Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant) for Angular SPA and Resource Server (JWT) for REST integration<h1>Introduction</h1>
<p>See <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-spring-security-6-oauth2_21.html">Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant) for Angular SPA</a></p>
<p>See <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-spring-security-6-oauth2.html">Spring Boot 3 and Keycloak with Oauth2 Resource Server (JWT) for REST integration</a></p>
<p>Here we will combine these two, to let same REST API service both a Angular app and for direct REST integrations.</p>
<h1>Prerequisite</h1>
<ul>
<li>Java 17</li>
<li>Maven 3.6.3 or later</li>
<li>Spring 3.1.2</li>
<li>Spring Security 6.1.2</li>
<li>Keycloak. I will use the commercial version RH SSO 7.6.0</li>
<li>OAuth2 Authorization Code Grand <a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.1">https://datatracker.ietf.org/doc/html/rfc6749#section-4.1</a></li>
<li>OAuth2 Bearer Token - Authorization: Bearer <Access Token></li>
</ul>
<h1>Application</h1>
<h2>pom.xml</h2>
<pre><code class="java"
><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.2</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>se.mkk</groupId>
<artifactId>spring-boot-3-oauth2-login-resource-server-keycloak-angular</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-boot-3-oauth2-login-resource-server-keycloak-angular</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>
<!-- Developer Tools -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<!-- Test Support -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<!-- https://github.com/40devweb/mvn-ng-sb/blob/main/pom.xml -->
<!-- build Angular frontend resources -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<id>front-end install</id>
<goals>
<goal>exec</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<executable>npm</executable>
<arguments>
<argument>install</argument>
</arguments>
</configuration>
</execution>
<execution>
<id>front-end build</id>
<goals>
<goal>exec</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<executable>npm</executable>
<arguments>
<argument>run</argument>
<argument>build</argument>
</arguments>
</configuration>
</execution>
</executions>
<configuration>
<workingDirectory>${basedir}/frontend</workingDirectory>
</configuration>
</plugin>
<!-- Copy Angular resources to Spring Boot resource files -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<id>copy front-end assets</id>
<goals>
<goal>copy-resources</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<outputDirectory>${basedir}/src/main/resources/static</outputDirectory>
<resources>
<resource>
<directory>frontend/dist/frontend</directory>
<!--<excludes>
<exclude>index.html</exclude>
</excludes>-->
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>copy front-end assets to target</id>
<goals>
<goal>copy-resources</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<outputDirectory>${basedir}/target/classes/static</outputDirectory>
<resources>
<resource>
<directory>frontend/dist/frontend</directory>
<!--<excludes>
<exclude>index.html</exclude>
</excludes>-->
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
<!-- Clean resources templates -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.2.0</version>
<configuration>
<filesets>
<fileset>
<directory>${basedir}/src/main/resources/static</directory>
<followSymlinks>false</followSymlinks>
</fileset>
<fileset>
<directory>${basedir}/src/main/resources/templates</directory>
<includes>
<include>index.html</include>
</includes>
<followSymlinks>false</followSymlinks>
</fileset>
<fileset>
<directory>${basedir}/frontend/dist</directory>
<includes>
<include>**/*</include>
</includes>
<followSymlinks>false</followSymlinks>
</fileset>
</filesets>
</configuration>
</plugin>
</plugins>
</build>
</project>
</code></pre>
<h2>src/main/resources/application.properties</h2>
<pre><code class="java"
># OAuth2 Log In Spring Boot 2.x Property Mappings
# https://docs.spring.io/spring-security/reference/servlet/oauth2/login/core.html#oauth2login-boot-property-mappings
spring.security.oauth2.client.registration.keycloak.client-id=spring-boot3-oauth2-login
spring.security.oauth2.client.registration.keycloak.client-secret=jO09Uwhi8oxTL3QnTKtYZ20ByQvB2qA0
#spring.security.oauth2.client.registration.keycloak.client-authentication-method =
# org.springframework.security.oauth2.core.AuthorizationGrantType
spring.security.oauth2.client.registration.keycloak.authorization-grant-type=authorization_code
#spring.security.oauth2.client.registration.keycloak.authorization-grant-type=urn:ietf:params:oauth:grant-type:jwt-bearer
#spring.security.oauth2.client.registration.keycloak.redirect-uri =
spring.security.oauth2.client.registration.keycloak.scope=openid
#spring.security.oauth2.client.registration.keycloak.client-name =
#spring.security.oauth2.client.provider.keycloak.authorization-uri
#spring.security.oauth2.client.provider.keycloak.token-uri
#spring.security.oauth2.client.provider.keycloak.jwk-set-uri
spring.security.oauth2.client.provider.keycloak.issuer-uri=http://localhost:8180/auth/realms/demo
#spring.security.oauth2.client.provider.keycloak.user-info-uri
#spring.security.oauth2.client.provider.keycloak.user-info-authentication-method
spring.security.oauth2.client.provider.keycloak.user-name-attribute=preferred_username
# https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html
# https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.security
spring.security.oauth2.resourceserver.jwt.issuer-uri = http://localhost:8180/auth/realms/demo
#spring.security.oauth2.resourceserver.jwt.jwk-set-uri = http://localhost:8180/auth/realms/demo/protocol/openid-connect/certs
# https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
#server.servlet.session.cookie.domain
server.servlet.session.cookie.http-only = true
server.servlet.session.cookie.max-age = 3m
#server.servlet.session.cookie.name
#server.servlet.session.cookie.path
server.servlet.session.cookie.same-site = strict
server.servlet.session.cookie.secure = true
server.servlet.session.persistent = false
#server.servlet.session.store-dir
server.servlet.session.timeout = 3m
server.servlet.session.tracking-modes = cookie
</code></pre>
<h2>Spring Security Java Config</h2>
<pre><code class="java"
>package se.mkk.springboot3oauth2loginresourceserverkeycloakangular;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.http.ResponseEntity;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserRequest;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserService;
import org.springframework.security.oauth2.core.OAuth2AccessToken;
import org.springframework.security.oauth2.core.oidc.user.DefaultOidcUser;
import org.springframework.security.oauth2.core.oidc.user.OidcUser;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationConverter;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.logout.HeaderWriterLogoutHandler;
import org.springframework.security.web.authentication.logout.LogoutHandler;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter.Directive;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
import com.nimbusds.jose.util.JSONObjectUtils;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class OAuth2LoginSecurityConfig {
private final Logger log = LoggerFactory.getLogger(this.getClass());
@Autowired
private RestTemplateBuilder restTemplateBuilder;
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.sessionManagement(
// https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html
// https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
Customizer.withDefaults())
.authorizeHttpRequests(authorizeHttpRequests -> authorizeHttpRequests //
.requestMatchers("/login", "/logout").permitAll() //
// .requestMatchers("/api/users/roles").hasRole("USER") //
.anyRequest().authenticated()) //
.oauth2ResourceServer(oauth2 -> oauth2 //
.jwt(jwt -> jwt //
.jwtAuthenticationConverter(this.jwtAuthenticationConverter())))
.oauth2Login(oauth2 -> oauth2 //
.userInfoEndpoint(userInfo -> userInfo //
.oidcUserService(this.oidcUserService())))
.csrf(csrf -> csrf //
// https://docs.spring.io/spring-security/reference/servlet/exploits/csrf.html#csrf-token-repository-cookie
.ignoringRequestMatchers("/logout", "/api"))
.logout(logout -> logout //
.addLogoutHandler(new KeycloakLogoutHandler(restTemplateBuilder.build())) //
// https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html#clear-all-site-data
.addLogoutHandler(new HeaderWriterLogoutHandler(new ClearSiteDataHeaderWriter(Directive.ALL))));
return http.build();
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html#oauth2resourceserver-jwt-authorization-extraction
private JwtAuthenticationConverter jwtAuthenticationConverter() {
JwtAuthenticationConverter jwtAuthenticationConverter = new JwtAuthenticationConverter();
jwtAuthenticationConverter.setPrincipalClaimName("preferred_username");
jwtAuthenticationConverter.setJwtGrantedAuthoritiesConverter(new KeycloakAuthoritiesConverter());
return jwtAuthenticationConverter;
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-map-authorities-oauth2userservice
private OAuth2UserService<OidcUserRequest, OidcUser> oidcUserService() {
final OidcUserService delegate = new OidcUserService();
return (userRequest) -> {
// Delegate to the default implementation for loading a user
OidcUser oidcUser = delegate.loadUser(userRequest);
OAuth2AccessToken accessToken = userRequest.getAccessToken();
Collection<GrantedAuthority> mappedAuthorities = new HashSet<>();
// 1) Fetch the authority information from the protected resource using accessToken
// 2) Map the authority information to one or more GrantedAuthority's and add it to mappedAuthorities
try {
String[] chunks = accessToken.getTokenValue().split("\\.");
Base64.Decoder decoder = Base64.getUrlDecoder();
String header = new String(decoder.decode(chunks[0]));
String payload = new String(decoder.decode(chunks[1]));
Map<String, Object> claims = JSONObjectUtils.parse(payload);
mappedAuthorities = new KeycloakAuthoritiesConverter().convert(claims);
} catch (Exception e) {
log.error("Failed to map Authorities", e);
}
// 3) Create a copy of oidcUser but use the mappedAuthorities instead
oidcUser = new DefaultOidcUser(mappedAuthorities, oidcUser.getIdToken(), oidcUser.getUserInfo(),
"preferred_username");
return oidcUser;
};
}
// Spring OAuth2 uses default Scopes Not Roles for Authorization
// org.springframework.security.oauth2.server.resource.authentication.JwtGrantedAuthoritiesConverter
private class KeycloakAuthoritiesConverter implements Converter<Jwt, Collection<GrantedAuthority>> {
@Override
public Collection<GrantedAuthority> convert(Jwt jwt) {
return convert(jwt.getClaims());
}
public Collection<GrantedAuthority> convert(Map<String, Object> claims) {
Collection<GrantedAuthority> grantedAuthorities = new ArrayList<>();
for (String authority : getAuthorities(claims)) {
grantedAuthorities.add(new SimpleGrantedAuthority("ROLE_" + authority));
}
return grantedAuthorities;
}
private Collection<String> getAuthorities(Map<String, Object> claims) {
Object realm_access = claims.get("realm_access");
log.info("Retrieved realm_access {}", realm_access);
if (realm_access instanceof Map) {
Map<String, Object> map = castAuthoritiesToMap(realm_access);
Object roles = map.get("roles");
if (roles instanceof Collection) {
return castAuthoritiesToCollection(roles);
}
}
return Collections.emptyList();
}
@SuppressWarnings("unchecked")
private Map<String, Object> castAuthoritiesToMap(Object authorities) {
return (Map<String, Object>) authorities;
}
@SuppressWarnings("unchecked")
private Collection<String> castAuthoritiesToCollection(Object authorities) {
return (Collection<String>) authorities;
}
}
// OpenID Connect 1.0 Logout Does Not work for Angular app, since redirect will violate CORS (Reason: CORS header
// ‘Access-Control-Allow-Origin’ missing) and OidcClientInitiatedLogoutSuccessHandler ignores Spring Security CORS
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-oidc-logout
// https://github.com/simasch/vaadin-keycloak/blob/main/src/main/java/ch/martinelli/demo/keycloak/security/KeycloakLogoutHandler.java
private class KeycloakLogoutHandler implements LogoutHandler {
private final RestTemplate restTemplate;
public KeycloakLogoutHandler(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
@Override
public void logout(HttpServletRequest request, HttpServletResponse response, Authentication auth) {
logoutFromKeycloak((OidcUser) auth.getPrincipal());
}
private void logoutFromKeycloak(OidcUser user) {
// https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.6/html-single/securing_applications_and_services_guide/index#logout
String endSessionEndpoint = user.getIssuer() + "/protocol/openid-connect/logout";
UriComponentsBuilder builder = UriComponentsBuilder //
.fromUriString(endSessionEndpoint) //
.queryParam("id_token_hint", user.getIdToken().getTokenValue());
ResponseEntity<String> logoutResponse = restTemplate.getForEntity(builder.toUriString(), String.class);
if (logoutResponse.getStatusCode().is2xxSuccessful()) {
log.info("Successfully logged out from Keycloak");
} else {
log.error("Could not propagate logout to Keycloak");
}
}
}
}
</code></pre>
<h1>Source Code</h1>
<p>See <a href="https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-resource-server-keycloak-angular">https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-resource-server-keycloak-angular</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-54681173127082681362023-08-21T22:48:00.004+02:002023-08-24T11:57:19.224+02:00Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant) for Angular SPA<h1>Introduction</h1>
<p>See <a href="https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-spring-security-6.html">https://magnus-k-karlsson.blogspot.com/2023/08/spring-boot-3-spring-security-6.html</a></p>
<h1>Prerequisite</h1>
<ul>
<li>Java 17</li>
<li>Maven 3.6.3 or later</li>
<li>Spring 3.1.2</li>
<li>Spring Security 6.1.2</li>
<li>Keycloak. I will use the commercial version RH SSO 7.6.0</li>
<li>OAuth2 Authorization Code Grand <a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.1">https://datatracker.ietf.org/doc/html/rfc6749#section-4.1</a></li>
</ul>
<h1>Application</h1>
<h2>Spring Security Session Management</h2>
<p>"The main point to take on board here is that security is stateful. You can’t have a secure, stateless application."</p>
<p>"TCP and SSL are stateful so your system is stateful whether you knew it or not"</p>
<p>Rob Winch Spring Exchange 2014 <a href="https://spring.io/guides/tutorials/spring-security-and-angular-js/">https://spring.io/guides/tutorials/spring-security-and-angular-js/</a></p>
<p><a href="https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html">https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html</a></p>
<p>The most important configuration is done in <b>src/main/resources/application.properties</b></p>
<pre><code class="java"
># https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
#server.servlet.session.cookie.domain
server.servlet.session.cookie.http-only = true
server.servlet.session.cookie.max-age = 3m
#server.servlet.session.cookie.name
#server.servlet.session.cookie.path
server.servlet.session.cookie.same-site = strict
server.servlet.session.cookie.secure = true
server.servlet.session.persistent = false
#server.servlet.session.store-dir
server.servlet.session.timeout = 3m
server.servlet.session.tracking-modes = cookie
</code></pre>
<h2>Application Logout and OpenID Connect 1.0 Logout</h2>
<p>Application logout <b>POST /logout</b></p>
<p><a href="https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html">https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html</a></p>
<p>OpenID Connect 1.0 Logout does <b>Not</b> work for SPA application since sending 302 with different Location other than Server will render CORS Error.</p>
<p><a href="https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-oidc-logout">https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-oidc-logout</a></p>
<p>Soo you need to implement custom <b>LogoutHandler</b></p>
<pre><code class="java"
>package se.mkk.springboot3oauth2loginkeycloakangular;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.http.ResponseEntity;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserRequest;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserService;
import org.springframework.security.oauth2.core.OAuth2AccessToken;
import org.springframework.security.oauth2.core.oidc.user.DefaultOidcUser;
import org.springframework.security.oauth2.core.oidc.user.OidcUser;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.web.SecurityFilterChain;
import org.springframework.security.web.authentication.logout.HeaderWriterLogoutHandler;
import org.springframework.security.web.authentication.logout.LogoutHandler;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter;
import org.springframework.security.web.header.writers.ClearSiteDataHeaderWriter.Directive;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
import com.nimbusds.jose.util.JSONObjectUtils;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class OAuth2LoginSecurityConfig {
private final Logger log = LoggerFactory.getLogger(this.getClass());
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.sessionManagement(
// https://docs.spring.io/spring-security/reference/servlet/authentication/session-management.html
// https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.server
Customizer.withDefaults())
.authorizeHttpRequests(authorizeHttpRequests -> authorizeHttpRequests //
.requestMatchers("/login", "/logout").permitAll() //
// .requestMatchers("/api/users/roles").hasRole("USER") //
.anyRequest().authenticated()) //
.oauth2Login(oauth2 -> oauth2 //
.userInfoEndpoint(userInfo -> userInfo //
.oidcUserService(this.oidcUserService())))
.csrf(csrf -> csrf //
// https://docs.spring.io/spring-security/reference/servlet/exploits/csrf.html#csrf-token-repository-cookie
.ignoringRequestMatchers("/logout", "/api"))
.logout(logout -> logout //
.addLogoutHandler(new KeycloakLogoutHandler(restTemplateBuilder.build())) //
// https://docs.spring.io/spring-security/reference/servlet/authentication/logout.html#clear-all-site-data
.addLogoutHandler(new HeaderWriterLogoutHandler(new ClearSiteDataHeaderWriter(Directive.ALL))));
return http.build();
}
@Autowired
private RestTemplateBuilder restTemplateBuilder;
// OpenID Connect 1.0 Logout Does Not work for Angular app, since redirect will violate CORS (Reason: CORS header
// ‘Access-Control-Allow-Origin’ missing) and OidcClientInitiatedLogoutSuccessHandler ignores Spring Security CORS
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-oidc-logout
// https://github.com/simasch/vaadin-keycloak/blob/main/src/main/java/ch/martinelli/demo/keycloak/security/KeycloakLogoutHandler.java
private class KeycloakLogoutHandler implements LogoutHandler {
private final RestTemplate restTemplate;
public KeycloakLogoutHandler(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
@Override
public void logout(HttpServletRequest request, HttpServletResponse response, Authentication auth) {
logoutFromKeycloak((OidcUser) auth.getPrincipal());
}
private void logoutFromKeycloak(OidcUser user) {
// https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.6/html-single/securing_applications_and_services_guide/index#logout
String endSessionEndpoint = user.getIssuer() + "/protocol/openid-connect/logout";
UriComponentsBuilder builder = UriComponentsBuilder //
.fromUriString(endSessionEndpoint) //
.queryParam("id_token_hint", user.getIdToken().getTokenValue());
ResponseEntity<String> logoutResponse = restTemplate.getForEntity(builder.toUriString(), String.class);
if (logoutResponse.getStatusCode().is2xxSuccessful()) {
log.info("Successfully logged out from Keycloak");
} else {
log.error("Could not propagate logout to Keycloak");
}
}
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-map-authorities-oauth2userservice
private OAuth2UserService<OidcUserRequest, OidcUser> oidcUserService() {
final OidcUserService delegate = new OidcUserService();
return (userRequest) -> {
// Delegate to the default implementation for loading a user
OidcUser oidcUser = delegate.loadUser(userRequest);
OAuth2AccessToken accessToken = userRequest.getAccessToken();
Collection<GrantedAuthority> mappedAuthorities = new HashSet<>();
// 1) Fetch the authority information from the protected resource using accessToken
// 2) Map the authority information to one or more GrantedAuthority's and add it to mappedAuthorities
try {
String[] chunks = accessToken.getTokenValue().split("\\.");
Base64.Decoder decoder = Base64.getUrlDecoder();
String header = new String(decoder.decode(chunks[0]));
String payload = new String(decoder.decode(chunks[1]));
Map<String, Object> claims = JSONObjectUtils.parse(payload);
mappedAuthorities = new KeycloakAuthoritiesConverter().convert(claims);
} catch (Exception e) {
log.error("Failed to map Authorities", e);
}
// 3) Create a copy of oidcUser but use the mappedAuthorities instead
oidcUser = new DefaultOidcUser(mappedAuthorities, oidcUser.getIdToken(), oidcUser.getUserInfo(),
"preferred_username");
return oidcUser;
};
}
// Spring OAuth2 uses default Scopes Not Roles for Authorization
// org.springframework.security.oauth2.server.resource.authentication.JwtGrantedAuthoritiesConverter
private class KeycloakAuthoritiesConverter implements Converter<Jwt, Collection<GrantedAuthority>> {
@Override
public Collection<GrantedAuthority> convert(Jwt jwt) {
return convert(jwt.getClaims());
}
public Collection<GrantedAuthority> convert(Map<String, Object> claims) {
Collection<GrantedAuthority> grantedAuthorities = new ArrayList<>();
for (String authority : getAuthorities(claims)) {
grantedAuthorities.add(new SimpleGrantedAuthority("ROLE_" + authority));
}
return grantedAuthorities;
}
private Collection<String> getAuthorities(Map<String, Object> claims) {
Object realm_access = claims.get("realm_access");
log.info("Retrieved realm_access {}", realm_access);
if (realm_access instanceof Map) {
Map<String, Object> map = castAuthoritiesToMap(realm_access);
Object roles = map.get("roles");
if (roles instanceof Collection) {
return castAuthoritiesToCollection(roles);
}
}
return Collections.emptyList();
}
@SuppressWarnings("unchecked")
private Map<String, Object> castAuthoritiesToMap(Object authorities) {
return (Map<String, Object>) authorities;
}
@SuppressWarnings("unchecked")
private Collection<String> castAuthoritiesToCollection(Object authorities) {
return (Collection<String>) authorities;
}
}
}
</code></pre>
<h1>Source code</h1>
<p><a href="https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak-angular">https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak-angular</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-43294054693519114052023-08-19T07:44:00.009+02:002023-08-24T09:19:27.245+02:00Spring Boot 3 and Keycloak with Oauth2 Resource Server (JWT) for REST integration<h1>Introduction</h1>
<p>Spring Security OAuth2 Login does NOT support authentication with Access Token that you might first think. </p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXJwNZsqNRqkO83e4Vw5_-fwiolu4i0QPIYYyVLZ9-giRLzHedNEV23bykPCcucQfWatxs5mCnm1Mz9MASBI1eDS_MDQaXiHyVuy11S7qZMQQhydo3ZDNlofvphOeV-N1TDVTYuHlvHfaZqVDUv0ACTZ3Gw2CkU8IstMzTOcZ6jBYGgSYJ8w9zSqp4sNA/s1188/Screenshot%20from%202023-08-19%2007-18-15.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="600" data-original-height="545" data-original-width="1188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXJwNZsqNRqkO83e4Vw5_-fwiolu4i0QPIYYyVLZ9-giRLzHedNEV23bykPCcucQfWatxs5mCnm1Mz9MASBI1eDS_MDQaXiHyVuy11S7qZMQQhydo3ZDNlofvphOeV-N1TDVTYuHlvHfaZqVDUv0ACTZ3Gw2CkU8IstMzTOcZ6jBYGgSYJ8w9zSqp4sNA/s600/Screenshot%20from%202023-08-19%2007-18-15.png"/></a></div>
<p><a href="https://docs.spring.io/spring-security/reference/servlet/oauth2/client/index.html">https://docs.spring.io/spring-security/reference/servlet/oauth2/client/index.html</a></p>
<p>The JWT Bearer is something different, that I have seen rarely used</p>
<pre><code class="java"
> POST /token.oauth2 HTTP/1.1
Host: as.example.com
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code&
code=n0esc3NRze7LTCu7iYzS6a5acc3f0ogp4&
client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3A
client-assertion-type%3Ajwt-bearer&
client_assertion=eyJhbGciOiJSUzI1NiIsImtpZCI6IjIyIn0.
eyJpc3Mi[...omitted for brevity...].
cC4hiUPo[...omitted for brevity...]
</code></pre>
<p>What you normally do, to access a OAuth2 protected resources is</p>
<pre><code class="java"
>GET / HTTP/1.1
Authorization: Bearer some-token-value # Resource Server will process this
</code></pre>
<p>And to setup backend for that you need Spring Security 6 OAuth2 Resource Server.</p>
<h1>Prerequisite</h1>
<ul>
<li>Java 17</li>
<li>Maven 3.6.3 or later</li>
<li>Spring 3.1.2</li>
<li>Spring Security 6.1.2 Resource Server</li>
<li>Keycloak. I will use the commercial version RH SSO 7.6.0</li>
<li>OAuth2 Resource Owner Password Credentials Grant <a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.3">https://datatracker.ietf.org/doc/html/rfc6749#section-4.3</a></li>
<li>Not neccessary but convenient jq - Command-line JSON processor (on Fedora $ sudo dnf install jq)</li>
</ul>
<h1>Application</h1>
<h2>pom.xml</h2>
<pre><code class="java"
><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.2</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>se.mkk</groupId>
<artifactId>spring-boot-3-oauth2-resource-server-keycloak</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-boot-3-oauth2-resource-server-keycloak</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
</code></pre>
<h2>src/main/resources/application.properties</h2>
<p>jwk-set-uri is not neccessary since spring security reads http://localhost:8180/auth/realms/demo/.well-known/openid-configuration.</p>
<pre><code class="java"
># https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html
# https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.application-properties.security
spring.security.oauth2.resourceserver.jwt.issuer-uri = http://localhost:8180/auth/realms/demo
#spring.security.oauth2.resourceserver.jwt.jwk-set-uri = http://localhost:8180/auth/realms/demo/protocol/openid-connect/certs
</code></pre>
<h2>Java code</h2>
<pre><code class="java"
>package se.mkk.springboot3oauth2resourceserverkeycloak;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
</code></pre>
<p>Simple REST endpoint</p>
<pre><code class="java"
>package se.mkk.springboot3oauth2resourceserverkeycloak;
import java.security.Principal;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
//import org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken;
import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import jakarta.servlet.http.HttpServletRequest;
@RestController
@RequestMapping("/api/users")
public class UserController {
@GetMapping
public Map<String, String> getUser(HttpServletRequest request, Principal principal) {
Map<String, String> rtn = new LinkedHashMap<>();
rtn.put("request.getRemoteUser()", request.getRemoteUser());
rtn.put("request.isUserInRole(\"USER\")", Boolean.toString(request.isUserInRole("USER")));
rtn.put("request.getUserPrincipal().getClass()", request.getUserPrincipal().getClass().getName());
rtn.put("principal.getClass().getName()", principal.getClass().getName());
rtn.put("principal.getName()", principal.getName());
if (principal instanceof JwtAuthenticationToken token) {
List<String> authorities = token.getAuthorities().stream()
.map(grantedAuthority -> grantedAuthority.getAuthority()).toList();
rtn.put("JwtAuthenticationToken.getAuthorities()", authorities.toString());
}
return rtn;
}
}
</code></pre>
<p>The OAuth2 Resource Server code. To make it work with Keycloak we need <b>2 adjustment</b>.</p>
<ol>
<li><b>Change username to preferred_username</b> - jwtAuthenticationConverter.setPrincipalClaimName("preferred_username")</li>
<li><b>Read Roles Not Scopes</b> - jwtAuthenticationConverter.setJwtGrantedAuthoritiesConverter(new KeycloakAuthoritiesConverter());</li>
</ol>
<pre><code class="java"
>package se.mkk.springboot3oauth2resourceserverkeycloak;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.Map;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationConverter;
import org.springframework.security.web.SecurityFilterChain;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class OAuth2ResourceServerSecurityConfig {
// https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html#oauth2resourceserver-jwt-sansboot
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.authorizeHttpRequests(authorize -> authorize //
.anyRequest().authenticated()) //
.oauth2ResourceServer(oauth2 -> oauth2 //
.jwt(jwt -> jwt //
.jwtAuthenticationConverter(this.jwtAuthenticationConverter())));
return http.build();
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html#oauth2resourceserver-jwt-authorization-extraction
private JwtAuthenticationConverter jwtAuthenticationConverter() {
JwtAuthenticationConverter jwtAuthenticationConverter = new JwtAuthenticationConverter();
jwtAuthenticationConverter.setPrincipalClaimName("preferred_username");
jwtAuthenticationConverter.setJwtGrantedAuthoritiesConverter(new KeycloakAuthoritiesConverter());
return jwtAuthenticationConverter;
}
// Spring OAuth2 uses default Scopes Not Roles for Authorization
// org.springframework.security.oauth2.server.resource.authentication.JwtGrantedAuthoritiesConverter
private class KeycloakAuthoritiesConverter implements Converter<Jwt, Collection<GrantedAuthority>> {
@Override
public Collection<GrantedAuthority> convert(Jwt jwt) {
return convert(jwt.getClaims());
}
public Collection<GrantedAuthority> convert(Map<String, Object> claims) {
Collection<GrantedAuthority> grantedAuthorities = new ArrayList<>();
for (String authority : getAuthorities(claims)) {
grantedAuthorities.add(new SimpleGrantedAuthority("ROLE_" + authority));
}
return grantedAuthorities;
}
private Collection<String> getAuthorities(Map<String, Object> claims) {
Object realm_access = claims.get("realm_access");
if (realm_access instanceof Map) {
Map<String, Object> map = castAuthoritiesToMap(realm_access);
Object roles = map.get("roles");
if (roles instanceof Collection) {
return castAuthoritiesToCollection(roles);
}
}
return Collections.emptyList();
}
@SuppressWarnings("unchecked")
private Map<String, Object> castAuthoritiesToMap(Object authorities) {
return (Map<String, Object>) authorities;
}
@SuppressWarnings("unchecked")
private Collection<String> castAuthoritiesToCollection(Object authorities) {
return (Collection<String>) authorities;
}
}
}
</code></pre>
<h1>Test</h1>
<p>Get Access Token from Keycloak</p>
<pre><code class="java"
>$ ACCESS_TOKEN=$(curl -s -X POST \
-H 'Content-Type: application/x-www-form-urlencoded' \
-u 'spring-boot3-oauth2-login:jO09Uwhi8oxTL3QnTKtYZ20ByQvB2qA0' \
http://localhost:8180/auth/realms/demo/protocol/openid-connect/token \
-d "grant_type=password&username=john&password=changeit" | jq -r .access_token)
</code></pre>
<p>Call REST api</p>
<pre><code class="java"
>$ curl -v -X GET -H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $ACCESS_TOKEN" \
http://localhost:8080/api/users | jq .
...
{
"request.getRemoteUser()": "john",
"request.isUserInRole(\"USER\")": "true",
"request.getUserPrincipal().getClass()": "org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken",
"principal.getClass().getName()": "org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken",
"principal.getName()": "john",
"JwtAuthenticationToken.getAuthorities()": "[ROLE_offline_access, ROLE_default-roles-demo, ROLE_uma_authorization, ROLE_USER]"
}
</code></pre>
<h1>Source Code</h1>
<p><a href="https://github.com/magnuskkarlsson/spring-boot-3-oauth2-resource-server-keycloak">https://github.com/magnuskkarlsson/spring-boot-3-oauth2-resource-server-keycloak</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-88112008838523199952023-08-18T07:30:00.017+02:002023-08-24T11:56:22.374+02:00Spring Boot 3 and Keycloak with Oauth2 Log in (Authorization Code Grant)<h1>Prerequisite</h1>
<ul>
<li>Java 17</li>
<li>Maven 3.6.3 or later</li>
<li>Spring 3.1.2</li>
<li>Spring Security 6.1.2</li>
<li>Keycloak. I will use the commercial version RH SSO 7.6.0</li>
<li>OAuth2 Authorization Code Grand <a href="https://datatracker.ietf.org/doc/html/rfc6749#section-4.1">https://datatracker.ietf.org/doc/html/rfc6749#section-4.1</a></li>
</ul>
<h1>RH SSO/Keycloak</h1>
<p>Download, unzip and create initial admin user and finally start at <a href="http://127.0.0.1:8180/">http://127.0.0.1:8180/</a>. You could also use a Docker container.</p>
<pre><code class="bash"
>$ ./add-user-keycloak.sh -u admin
$ ./standalone.sh -Djboss.socket.binding.port-offset=100
</code></pre>
<p>In Keycloak create</p>
<ul>
<li>New Realm demo</li>
<li>Role USER</li>
<li>User john with password</li>
<li>Assign role USER to user john</li>
<li>Create OIDC client with Client ID spring-boot3-oauth2-login and Root URL http://localhost:8080/</li>
</ul>
<h1>Application</h1>
<h2>pom.xml</h2>
<pre><code class="xml"
><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.2</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>se.mkk</groupId>
<artifactId>spring-boot-3-oauth2-login-keycloak</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-boot-3-oauth2-login-keycloak</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
</code></pre>
<h2>src/main/resources/application.properties</h2>
<pre><code class="bash"
># OAuth2 Log In Spring Boot 2.x Property Mappings
# https://docs.spring.io/spring-security/reference/servlet/oauth2/login/core.html#oauth2login-boot-property-mappings
spring.security.oauth2.client.registration.keycloak.client-id = spring-boot3-oauth2-login
spring.security.oauth2.client.registration.keycloak.client-secret = CHANGEME!!!
#spring.security.oauth2.client.registration.keycloak.client-authentication-method =
spring.security.oauth2.client.registration.keycloak.authorization-grant-type = authorization_code
#spring.security.oauth2.client.registration.keycloak.redirect-uri =
spring.security.oauth2.client.registration.keycloak.scope = openid
#spring.security.oauth2.client.registration.keycloak.client-name =
#spring.security.oauth2.client.provider.keycloak.authorization-uri
#spring.security.oauth2.client.provider.keycloak.token-uri
#spring.security.oauth2.client.provider.keycloak.jwk-set-uri
spring.security.oauth2.client.provider.keycloak.issuer-uri = http://localhost:8180/auth/realms/demo
#spring.security.oauth2.client.provider.keycloak.user-info-uri
#spring.security.oauth2.client.provider.keycloak.user-info-authentication-method
spring.security.oauth2.client.provider.keycloak.user-name-attribute = preferred_username
</code></pre>
<h2>Simple REST API</h2>
<pre><code class="java"
>package se.mkk.springboot3oauth2loginkeycloak;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
</code></pre>
<pre><code class="java"
>package se.mkk.springboot3oauth2loginkeycloak;
import java.security.Principal;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import jakarta.servlet.http.HttpServletRequest;
@RestController
@RequestMapping("/api/users")
public class UserController {
@GetMapping
public Map<String, String> getUser(HttpServletRequest request, Principal principal) {
Map<String, String> rtn = new LinkedHashMap<>();
rtn.put("request.getRemoteUser()", request.getRemoteUser());
rtn.put("request.isUserInRole(\"USER\")", Boolean.toString(request.isUserInRole("USER")));
rtn.put("request.getUserPrincipal().getClass()", request.getUserPrincipal().getClass().getName());
rtn.put("principal.getClass().getName()", principal.getClass().getName());
rtn.put("principal.getName()", principal.getName());
if (principal instanceof OAuth2AuthenticationToken token) {
List<String> authorities = token.getAuthorities().stream()
.map(grantedAuthority -> grantedAuthority.getAuthority()).toList();
rtn.put("OAuth2AuthenticationToken.getAuthorities()", authorities.toString());
}
return rtn;
}
}
</code></pre>
<h2>OAuth2 Log in</h2>
<pre><code class="java"
>ppackage se.mkk.springboot3oauth2loginkeycloak;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.Map;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.convert.converter.Converter;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserRequest;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserService;
import org.springframework.security.oauth2.core.OAuth2AccessToken;
import org.springframework.security.oauth2.core.oidc.user.DefaultOidcUser;
import org.springframework.security.oauth2.core.oidc.user.OidcUser;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.web.SecurityFilterChain;
import com.nimbusds.jose.util.JSONObjectUtils;
@Configuration
@EnableWebSecurity
@EnableMethodSecurity
public class OAuth2LoginSecurityConfig {
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/core.html#oauth2login-provide-securityfilterchain-bean
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http //
.authorizeHttpRequests(authorize -> authorize //
.anyRequest().authenticated()) //
.oauth2Login(oauth2 -> oauth2 //
.userInfoEndpoint(userInfo -> userInfo //
.oidcUserService(this.oidcUserService())));
return http.build();
}
// https://docs.spring.io/spring-security/reference/servlet/oauth2/login/advanced.html#oauth2login-advanced-map-authorities-oauth2userservice
private OAuth2UserService<OidcUserRequest, OidcUser> oidcUserService() {
final OidcUserService delegate = new OidcUserService();
return (userRequest) -> {
// Delegate to the default implementation for loading a user
OidcUser oidcUser = delegate.loadUser(userRequest);
OAuth2AccessToken accessToken = userRequest.getAccessToken();
Collection<GrantedAuthority> mappedAuthorities = new HashSet<>();
// 1) Fetch the authority information from the protected resource using accessToken
// 2) Map the authority information to one or more GrantedAuthority's and add it to mappedAuthorities
try {
String[] chunks = accessToken.getTokenValue().split("\\.");
Base64.Decoder decoder = Base64.getUrlDecoder();
String header = new String(decoder.decode(chunks[0]));
String payload = new String(decoder.decode(chunks[1]));
Map<String, Object> claims = JSONObjectUtils.parse(payload);
mappedAuthorities = new KeycloakAuthoritiesConverter().convert(claims);
} catch (Exception e) {
e.printStackTrace();
}
// 3) Create a copy of oidcUser but use the mappedAuthorities instead
oidcUser = new DefaultOidcUser(mappedAuthorities, oidcUser.getIdToken(), oidcUser.getUserInfo(),
"preferred_username");
return oidcUser;
};
}
// Spring OAuth2 uses default Scopes Not Roles for Authorization
// org.springframework.security.oauth2.server.resource.authentication.JwtGrantedAuthoritiesConverter
private class KeycloakAuthoritiesConverter implements Converter<Jwt, Collection<GrantedAuthority>> {
@Override
public Collection<GrantedAuthority> convert(Jwt jwt) {
return convert(jwt.getClaims());
}
public Collection<GrantedAuthority> convert(Map<String, Object> claims) {
Collection<GrantedAuthority> grantedAuthorities = new ArrayList<>();
for (String authority : getAuthorities(claims)) {
grantedAuthorities.add(new SimpleGrantedAuthority("ROLE_" + authority));
}
return grantedAuthorities;
}
private Collection<String> getAuthorities(Map<String, Object> claims) {
Object realm_access = claims.get("realm_access");
if (realm_access instanceof Map) {
Map<String, Object> map = castAuthoritiesToMap(realm_access);
Object roles = map.get("roles");
if (roles instanceof Collection) {
return castAuthoritiesToCollection(roles);
}
}
return Collections.emptyList();
}
@SuppressWarnings("unchecked")
private Map<String, Object> castAuthoritiesToMap(Object authorities) {
return (Map<String, Object>) authorities;
}
@SuppressWarnings("unchecked")
private Collection<String> castAuthoritiesToCollection(Object authorities) {
return (Collection<String>) authorities;
}
}
}
</code></pre>
<p>And now run</p>
<pre><code class="bash"
>$ mvn clean install spring-boot:run
</code></pre>
<p>And login and call REST endpoint</p>
<pre><code class="java"
>request.getRemoteUser() "john"
request.isUserInRole("USER") "true"
request.getUserPrincipal().getClass() "org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken"
principal.getClass().getName() "org.springframework.security.oauth2.client.authentication.OAuth2AuthenticationToken"
principal.getName() "john"
OAuth2AuthenticationToken.getAuthorities() "[ROLE_USER, ROLE_default-roles-demo, ROLE_offline_access, ROLE_uma_authorization]"
</code></pre>
<h1>Summary</h1>
<p>You also want to configure Spring <b>Session</b>, <b>Logout</b> and <b>CSRF</b>.</p>
<p>Source code <a href="https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak">https://github.com/magnuskkarlsson/spring-boot-3-oauth2-login-keycloak</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-68741831487410908592023-07-08T21:55:00.000+02:002023-07-08T21:55:02.913+02:00Building Spring Boot 3 Jar with Angular 15<p><a href="https://github.com/magnuskkarlsson/spring-boot3-angular15/blob/main/HELP.md" target="_blank"> https://github.com/magnuskkarlsson/spring-boot3-angular15/blob/main/HELP.md</a></p><p> </p>Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-6231530269719812862023-01-14T13:44:00.003+01:002023-01-14T13:44:47.131+01:00Fedora 37 Getting Started with Virtualization (KVM)<pre><code class="bash"
>$ sudo dnf groupinfo virtualization
Last metadata expiration check: 0:03:45 ago on Sat 14 Jan 2023 10:51:52 AM CET.
Group: Virtualization
Description: These packages provide a graphical virtualization environment.
Mandatory Packages:
virt-install
Default Packages:
libvirt-daemon-config-network
libvirt-daemon-kvm
qemu-kvm
virt-manager
virt-viewer
Optional Packages:
guestfs-tools
libguestfs-tools
python3-libguestfs
virt-top
$ sudo dnf install @virtualization
$ sudo systemctl enable --now libvirtd
$ lsmod | grep kvm
kvm_intel 389120 0
kvm 1122304 1 kvm_intel
irqbypass 16384 1 kvm
</code></pre>
<p><a href="https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-virtualization/">https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-virtualization/</a></p>Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-53724110337173550902023-01-14T08:16:00.007+01:002023-01-14T08:16:57.680+01:00Fedora 37 How to Turn off System Beep / Bell Terminal Sound<pre><code class="bash"
>$ less /proc/modules | grep pcspkr
pcspkr 16384 0 - Live 0x0000000000000000
$ lsmod | grep pcspkr
$ modinfo pcspkr
filename: /lib/modules/6.0.18-300.fc37.x86_64/kernel/drivers/input/misc/pcspkr.ko.xz
alias: platform:pcspkr
license: GPL
description: PC Speaker beeper driver
...
$ sudo modprobe -r -v pcspkr
</code></pre>
<p><a href="https://www.cyberciti.biz/faq/rhel-fedora-turn-off-bell-beep-sound/">https://www.cyberciti.biz/faq/rhel-fedora-turn-off-bell-beep-sound/</a></p>
<p><a href="https://www.cyberciti.biz/faq/howto-display-list-of-modules-or-device-drivers-in-the-linux-kernel/">https://www.cyberciti.biz/faq/howto-display-list-of-modules-or-device-drivers-in-the-linux-kernel/</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-66028203960102991572023-01-14T07:59:00.005+01:002023-01-14T07:59:58.634+01:00Fedora 37 don't Group Application when Alt + Tab<p>This describes how to enable classic Windows Alt-Tab or not Group Application when switching windows.</p>
<p>In Fedora 37 the rpm package gnome-shell-extension-alternate-tab is gone.</p>
<p><b>Open Settings -> Keyboard -> Keyboard Shortcuts: View and Customize ShortCuts</b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__3KfbdlsEdxtz4I5T_MuYv_eKslMDeOGyGZYX-xqefmBYI6kDgMXkvgjzHFQ0gql-KhBqCp8yv9hxuGpB5CEiTm1CFJvbTMJdfxmUrIP7-OYbhzYOUJgCVL7ZjE38PsCKlQ11pmaN3tUmmUOLUL3Mq43QPvd9ZQkU62px4YMNn1f-uRAOnbCPM6W/s1327/Screenshot%20from%202023-01-14%2007-52-06.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="600" data-original-height="1327" data-original-width="1305" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__3KfbdlsEdxtz4I5T_MuYv_eKslMDeOGyGZYX-xqefmBYI6kDgMXkvgjzHFQ0gql-KhBqCp8yv9hxuGpB5CEiTm1CFJvbTMJdfxmUrIP7-OYbhzYOUJgCVL7ZjE38PsCKlQ11pmaN3tUmmUOLUL3Mq43QPvd9ZQkU62px4YMNn1f-uRAOnbCPM6W/s600/Screenshot%20from%202023-01-14%2007-52-06.png"/></a></div>
<p>Click on <b>Switch Windows</b> and set new shortcut by pressing Alt-Tab.</p>
<p>Reference: <a href="https://blogs.gnome.org/fmuellner/2018/10/11/the-future-of-alternatetab-and-why-you-need-not-worry/">https://blogs.gnome.org/fmuellner/2018/10/11/the-future-of-alternatetab-and-why-you-need-not-worry/</a></p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-50584030845501952962023-01-12T18:46:00.007+01:002023-01-13T14:04:04.203+01:00Updating BIOS and Firmware for DELL and Fedora 37<h2>Introduction</h2>
<p>These are excellent guides</p>
<ul>
<li><a href="https://dellwindowsreinstallationguide.com/uefi-bios-update-usb/">https://dellwindowsreinstallationguide.com/uefi-bios-update-usb/</a></li>
<li><a href="https://dellwindowsreinstallationguide.com/linux-vendor-firmware-service-uefi-bios-update-ubuntu/">https://dellwindowsreinstallationguide.com/linux-vendor-firmware-service-uefi-bios-update-ubuntu/</a></li>
<li><a href="https://dellwindowsreinstallationguide.com/fedora-34/">https://dellwindowsreinstallationguide.com/fedora-34/</a></li>
<li><a href="https://dellwindowsreinstallationguide.com/fedora-36/">https://dellwindowsreinstallationguide.com/fedora-36/</a></li>
</ul>
<h2>To check BIOS version from Fedora</h2>
<pre><code class="bash"
>$ sudo dmidecode -s bios-version
1.23.0
</code></pre>
<pre><code class="bash"
>$ sudo dmidecode
# dmidecode 3.4
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.
...
BIOS Information
Vendor: Dell Inc.
Version: 1.23.0
Release Date: 07/06/2022
...
Characteristics:
PCI is supported
PNP is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
ACPI is supported
USB legacy is supported
Smart battery is supported
BIOS boot specification is supported
Function key-initiated network boot is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 1.23
...
</code></pre>
<p>Glossary</p>
<ul>
<li>Basic Input Output System (BIOS)</li>
<li>Unified Extensive Firmware Interface (UEFI)</li>
<li>System Management BIOS (SMBIOS)</li>
</ul>
<p>"In 2012 the BIOS was superseded with the much more advanced Unified Extensive Firmware Interface (UEFI)."</p>
<p>"Another term that gets often confused with BIOS is the System Management BIOS. The system management BIOS doesn't change unless you physically upgrade the motherboard or purchase a new computer. It is a reflection of the age of the hardware and the number of technologies made available."</p>
<h2>To check Firmware version from Fedora</h2>
<p>When updating BIOS, you do not update Firmware (FW), such as Solid State Drive (SSD), Thunderbolt Dock Firmware, etc</p>
<pre><code class="bash"
>$ sudo fwupdmgr get-devices
Dell Inc. XPS 15 9550
│
├─Core™ i7-6700HQ CPU @ 2.60GHz:
│ Device ID: 4bde70ba4e39b28f9eab1628f9dd6e6244c03027
│ Current version: 0x000000f0
│ Vendor: Intel
│ GUIDs: b9a2dd81-159e-5537-a7db-e7101d164d3f ← cpu
...
│ Device Flags: • Internal device
│
├─GM107M [GeForce GTX 960M] (XPS 15 9550):
...
</code></pre>
<p>"The Linux Vendor Firmware Service (LVFS) has been put together by Device Vendors or OEMs as a means for users to easily update their devices firmware using Linux. Dell and Lenovo in particular have been widely using the LVFS."</p>
<p><a href="https://fwupd.org/lvfs/devices/">https://fwupd.org/lvfs/devices/</a></p>
<h2>To Update BIOS and Firmware from Fedora with LVFS</h2>
<p>List all devices.</p>
<pre><code class="bash"
>$ sudo fwupdmgr refresh --force
$ sudo fwupdmgr get-devices
</code></pre>
<p>Check for updates for all devices.</p>
<pre><code class="bash"
>$ sudo fwupdmgr get-updates
</code></pre>
<p>Updates all devices.</p>
<pre><code class="bash"
>$ sudo fwupdmgr update
</code></pre>
<p>If your system isn't supported by the Linux Vendor Firmware Service (LVFS), then you must use BIOS Flash Update. Use a blank USB Flash Drive (FAT32 formatted) that contains the UEFI BIOS Update, that you download from Dell Support Website - https://www.dell.com/support/home/</p>
<p>Clear TPM <a href="https://www.dell.com/support/kbdoc/en-us/000184894/how-to-successfully-update-the-tpm-firmware-on-your-dell-computer">https://www.dell.com/support/kbdoc/en-us/000184894/how-to-successfully-update-the-tpm-firmware-on-your-dell-computer</a></p>
<pre><code class="bash"
>Update Error: Updating disabled due to TPM ownership
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-50000828242371309062022-11-02T22:48:00.002+01:002022-11-02T22:48:23.441+01:00OpenShift 4.6 Automation and Integration: Recovering Failed Worker Nodes<h2>Node Status</h2>
<pre><code class="bash"
>$ oc get nodes <NODE>
$ oc adm top node <NODE>
$ oc describe node <NODE> | grep -i taint
</code></pre>
<h2>OpenShift Taint Effects</h2>
<p>
3.6.1. Understanding taints and tolerations<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations</a>
</p>
<ul>
<li>PreferNoSchedule</li>
<li>NoSchedule</li>
<li>NoExecute</li>
</ul>
<pre><code class="bash"
>apiVersion: v1
kind: Node
metadata:
annotations:
machine.openshift.io/machine: openshift-machine-api/ci-ln-62s7gtb-f76d1-v8jxv-master-0
machineconfiguration.openshift.io/currentConfig: rendered-master-cdc1ab7da414629332cc4c3926e6e59c
...
spec:
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
</code></pre>
<h2>Worker Node Not Ready</h2>
<pre><code class="bash"
>$ oc describe node/worker01
...output omitted...
Taints: node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/not-ready:NoSchedule
...
Ready False ... KubeletNotReady [container runtime is down...
</code></pre>
<pre><code class="bash"
>$ ssh core@worker01 "sudo systemctl is-active crio"
$ ssh core@worker01 "sudo systemctl start crio"
$ oc describe node/worker01 | grep -i taints
</code></pre>
<h2>Worker Node Storage Exhaustion</h2>
<p>
3.6.1. Understanding taints and tolerations<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations</a>
</p>
<p>node.kubernetes.io/disk-pressure: The node has disk pressure issues. This corresponds to the node condition DiskPressure=True.</p>
<pre><code class="bash"
>$ oc describe node/worker01
...
Taints: disk-pressure:NoSchedule
disk-pressure:NoExecute
...
</code></pre>
<h2>Worker Node Capacity</h2>
<pre><code class="bash"
>$ oc get pod -o wide
NAME READY STATUS ... NODE ...
diskuser-4cfdd 0/1 Pending ... <none> ...
diskuser-ck4df 0/1 Evicted ... worker02 ...
$ oc describe node/worker01
...output omitted...
Taints: node.kubernetes.io/not-ready:NoSchedule
...
Conditions:
Type Status ... Reason ...
---- ------ ... ------ ...
DiskPressure True ... KubeletHasDiskPressure ...
</code></pre>
<h2>Worker Node Unreachable</h2>
<p>
3.6.1. Understanding taints and tolerations<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations</a>
</p>
<p>node.kubernetes.io/unreachable: The node is unreachable from the node controller. This corresponds to the node condition Ready=Unknown.</p>
<pre><code class="bash"
>$ ssh core@worker02 "sudo systemctl is-active kubelet"
$ ssh core@worker02 "sudo systemctl start kubelet"
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-61239384481507157992022-11-02T22:43:00.000+01:002022-11-02T22:43:06.086+01:00OpenShift 4.6 Automation and Integration: Kibana<h2>Filtering Queries</h2>
<p>
12.3. Kubernetes exported fields<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-exported-fields-kubernetes_cluster-logging-exported-fields" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-exported-fields-kubernetes_cluster-logging-exported-fields</a>
</p>
<p>These are the Kubernetes fields exported by the OpenShift Container Platform cluster logging available for searching from Elasticsearch and Kibana.</p>
<table border="1">
<tr>
<td>hostname</td>
<td>The hostname of OpenShift node that generated the message.</td>
</tr>
<tr>
<td>kubernetes.flat_labels</td>
<td>The label for the pod that generated the message. Format: key=value</td>
</tr>
<tr>
<td>kubernetes.container_name</td>
<td>The name of the container in Kubernetes.</td>
</tr>
<tr>
<td>kubernetes.namespace_name</td>
<td>The name of the namespace in Kubernetes.</td>
</tr>
<tr>
<td>kubernetes.pod_name</td>
<td>The name of the pod that generated the log message.</td>
</tr>
<tr>
<td>level</td>
<td>The log level of the message.</td>
</tr>
<tr>
<td>message</td>
<td>The actual log message.</td>
</tr>
</table>
<p>Example Lucene query:</p>
<pre><code class="bash"
>+kubernetes.namespace_name:"openshift-etcd" +message:elected
</code></pre>
<h2>Finding OpenShift Event Logs</h2>
<table>
<tr>
<td>kubernetes.event</td>
<td> </td>
</tr>
<tr>
<td>kubernetes.event.involvedObject.name</td>
<td>Resource name invloved in event.</td>
</tr>
<tr>
<td>kubernetes.event.involvedObject.namespace</td>
<td>Namespace of the resource name invloved in event.</td>
</tr>
<tr>
<td>kubernetes.event.reason</td>
<td>The reason for the event. Correspond to the values in the REASON column that displays in the output of the oc get events command.</td>
</tr>
<tr>
<td>kubernetes.event.type</td>
<td>The type of message, e.g. kubernetes.event.type:warning </td>
</tr>
</table>
<h2>Visualizing Time Series with Timelion</h2>
<p>
Timelion Tutorial – From Zero to Hero<br/>
<a href="https://www.elastic.co/blog/timelion-tutorial-from-zero-to-hero" target="_blank">https://www.elastic.co/blog/timelion-tutorial-from-zero-to-hero</a>
</p>
<pre><code class="bash"
>.es('+kubernetes.namespace_name:logging-query +message:200'),
.es('+kubernetes.namespace_name:logging-query +message:404'),
.es('+kubernetes.namespace_name:logging-query +message:500')
.es('+kubernetes.container_name:logger +message:500')
.divide(.es('+kubernetes.container_name:logger +message:*'))
.multiply(100)
.es('+kubernetes.container_name:logger +message:500').label(current),
.es(q='+kubernetes.container_name:logger +message:500', offset=-5m).label(previous)
</code></pre>
<h2>Troubleshooting cluster logging</h2>
<p>
Chapter 10. Troubleshooting cluster logging<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#troubleshooting-cluster-logging" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#troubleshooting-cluster-logging</a>
</p>
<pre><code class="bash"
>$ oc get -n openshift-logging clusterlogging instance -o yaml
apiVersion: logging.openshift.io/v1
kind: ClusterLogging
....
status:
...
logstore:
elasticsearchStatus:
- ShardAllocationEnabled: all
cluster:
activePrimaryShards: 5
activeShards: 5
initializingShards: 0
numDataNodes: 1
numNodes: 1
pendingTasks: 0
relocatingShards: 0
status: green
unassignedShards: 0
clusterName: elasticsearch
...
</code></pre>
<h3>Using Grafana</h3>
<p>Monitoring -> Dashboards:</p>
<p>
Dashboards: Kubernetes / Compute Resources / Node (Pods) <br/>
Namespace: openshift-logging
</p>
<h3>Using Kibana</h3>
<p>
Infra index<br/>
+kubernetes.namespace_name:openshift-logging +kubernetes.container_name:
</p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-82923960987974139432022-11-02T12:45:00.003+01:002022-11-02T14:06:31.826+01:00OpenShift 4.6 Automation and Integration: Cluster Logging<h2>Overview</h2>
<p>
1.1.8. About cluster logging components<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-about-components_cluster-logging" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-about-components_cluster-logging</a>
</p>
<p>The major components of cluster logging are:</p>
<h3>LogStore</h3>
<p>The logStore is the Elasticsearch cluster that</p>
<ul>
<li>Stores the logs into indexes.</li>
<li>Provides RBAC access to the logs.</li>
<li>Provides data redundancy.</li>
</ul>
<h3>Collection</h3>
<p>Implemented with Fluentd, By default, the log collector uses the following sources:</p>
<ul>
<li><b>journald</b> for all system logs</li>
<li><b>/var/log/containers/*.log</b> for all container logs</li>
</ul>
<p>The logging collector is deployed as a daemon set that deploys pods to each OpenShift Container Platform node.</p>
<h3>Visualization</h3>
<p>This is the UI component you can use to view logs, graphs, charts, and so forth. The current implementation is Kibana.</p>
<h3>Event Routing</h3>
<p>The Event Router is a pod that watches OpenShift Container Platform events so they can be collected by cluster logging. The Event Router collects events from all projects and writes them to STDOUT. Fluentd collects those events and forwards them into the OpenShift Container Platform Elasticsearch instance. Elasticsearch indexes the events to the infra index.</p>
<p>You must manually deploy the Event Router.</p>
<h2>Installing cluster logging</h2>
<p>
Chapter 2. Installing cluster logging
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-deploying
</p>
<h3>Install the OpenShift Elasticsearch Operator</h3>
<p><i>namespace: openshift-operators-redhat</i></p>
<h3>Install the Cluster Logging Operator</h3>
<p><i>namespace: openshift-logging</i></p>
<h3>Deploying a Cluster Logging Instance</h3>
<p>This default cluster logging configuration should support a wide array of environments.</p>
<pre><code class="bash"
>apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
name: "instance"
namespace: "openshift-logging"
spec:
managementState: "Managed"
logStore:
type: "elasticsearch"
retentionPolicy:
application:
maxAge: 1d
infra:
maxAge: 7d
audit:
maxAge: 7d
elasticsearch:
nodeCount: 3 5
storage:
storageClassName: "<storage-class-name>"
size: 200G
resources:
limits:
memory: "16Gi"
requests:
memory: "16Gi"
proxy:
resources:
limits:
memory: 256Mi
requests:
memory: 256Mi
redundancyPolicy: "SingleRedundancy"
visualization:
type: "kibana"
kibana:
replicas: 1
curation:
type: "curator"
curator:
schedule: "30 3 * * *"
collection:
logs:
type: "fluentd"
fluentd: {}
</code></pre>
<h3>Verify</h3>
<pre><code class="bash"
>$ oc get clusterlogging -n openshift-logging instance -o yaml
</code></pre>
<h3>Install the Event Router</h3>
<p>
7.1. Deploying and configuring the Event Router<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-eventrouter-deploy_cluster-logging-curator" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/logging/index#cluster-logging-eventrouter-deploy_cluster-logging-curator</a>
</p>
<h2>Creating Kibana Index Patterns</h2>
<p>
Index Pattern: <b>app-*</b><br/>
Time Filter Field Name: <b>@timestamp</b>
</p>
<p>
Index Pattern: <b>infra-*</b><br/>
Time Filter Field Name: <b>@timestamp</b>
</p>
<p>
Index Pattern: <b>audit-*</b><br/>
Time Filter Field Name: <b>@timestamp</b>
</p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-82330676671184202882022-11-02T11:47:00.001+01:002022-11-02T14:50:29.768+01:00OpenShift 4.6 Automation and Integration: Cluster Monitoring and Metrics<h2>Overview Monitoring</h2>
<p>
1.2. Understanding the monitoring stack<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#understanding-the-monitoring-stack_monitoring-overview" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#understanding-the-monitoring-stack_monitoring-overview</a>
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLz2HbqSLHoVwTgqptk0eZvp2aRrNBdMA_Kmhr7R5N8odrlebs3Eza57MBOg5YBHgsqVmZGUDyVqoIWCRdMXuU2jHfz6QA9W9Ra_SxUmLgrRxEutle9JwAHkwXOoM7JXt3s4QKVyaM4uQvE4ljveSkYPXkM8xz8G1Ly0vOpecd2-TbGeePydzceyYp/s1208/Screenshot%20from%202022-11-02%2011-30-26.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="400" data-original-height="1208" data-original-width="863" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLz2HbqSLHoVwTgqptk0eZvp2aRrNBdMA_Kmhr7R5N8odrlebs3Eza57MBOg5YBHgsqVmZGUDyVqoIWCRdMXuU2jHfz6QA9W9Ra_SxUmLgrRxEutle9JwAHkwXOoM7JXt3s4QKVyaM4uQvE4ljveSkYPXkM8xz8G1Ly0vOpecd2-TbGeePydzceyYp/s400/Screenshot%20from%202022-11-02%2011-30-26.png"/></a></div>
<h2>Alertmanager</h2>
<p>
5.7. Applying a custom Alertmanager configuration<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#applying-custom-alertmanager-configuration_managing-alerts" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#applying-custom-alertmanager-configuration_managing-alerts</a>
</p>
<pre><code class="bash"
>$ oc extract secret/alertmanager-main --to /tmp/ -n openshift-monitoring --confirm
</code></pre>
<p><b>OCP Web Console</b></p>
<p>Navigate to the <b>Administration -> Cluster Settings -> Global Configuration -> Alertmanager -> YAML</b>.</p>
<pre><code class="bash"
>global:
resolve_timeout: 5m
route:
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
receiver: default
routes:
- match:
alertname: Watchdog
repeat_interval: 5m
receiver: watchdog
receivers:
- name: default
- name: watchdog
</code></pre>
<p>Sending Alerts to Email</p>
<pre><code class="bash"
>global:
resolve_timeout: 5m
smtp_smarthost: "mail.mkk.se:25"
smtp_from: alerts@ocp4.mkk.se
smtp_auth_username: mail_username
smtp_auth_password: mail_password
smtp_require_tls: false
route:
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
receiver: default
routes:
- match:
alertname: Watchdog
repeat_interval: 5m
receiver: watchdog
- match:
severity: critical
receiver: email-notification
receivers:
- name: default
- name: watchdog
- name: email-notification
email_configs:
- to: ocp-admins@mkk.se
</code></pre>
<pre><code class="bash"
>$ oc set data secret/alertmanager-main -n openshift-monitoring --from-file=/tmp/alertmanager.yaml
$ oc logs -f -n openshift-monitoring alertmanager-main-0 -c alertmanager
</code></pre>
<h2>Grafana</h2>
<p>Grafana includes the following default dashboards:</p>
<table border="1">
<tr>
<td><b>etcd</b></td>
<td>Information on etcd in cluster.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Cluster</b></td>
<td>High-level view of cluster resources.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Namespace (Pods)</b></td>
<td>Resource usage for pods per namespace.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Namespace (Workloads)</b></td>
<td>Resource usage per namespace and then by workload type, such as deployment, daemonset, and statefulset.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Node (Pods)</b></td>
<td>Resource usage per node.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Pod</b></td>
<td>Resource usage for individual pods.</td>
</tr>
<tr>
<td><b>Kubernetes / Compute Resources / Workload</b></td>
<td>Resources usage per namespace, workload, and workload type.</td>
</tr>
<tr>
<td><b>Kubernetes / Networking/Cluster</b></td>
<td>Network usage in cluster</td>
</tr>
<tr>
<td><b>Prometheus</b></td>
<td>Information about prometheus-k8s pods running in the openshift-monitoring namespace.</td>
</tr>
<tr>
<td><b>USE Method / Cluster</b></td>
<td>USE, Utilization Saturation and Errors.</tr>
</table>
<h2>Persistent Storage</h2>
<h3>Configuring Prometheus Persistent Storage</h3>
<p>
2.8.2. Configuring a local persistent volume claim<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#configuring-a-local-persistent-volume-claim_configuring-the-monitoring-stack" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/monitoring/index#configuring-a-local-persistent-volume-claim_configuring-the-monitoring-stack</a>
</p>
<pre><code class="bash"
>apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-monitoring-config
namespace: openshift-monitoring
data:
config.yaml: |
prometheusK8s:
retention: 15d
volumeClaimTemplate:
spec:
storageClassName: local-storage
volumeMode: Filesystem
resources:
requests:
storage: 40Gi
</code></pre>
<h3>Configuring Alert Manager Persistent Storage</h3>
<pre><code class="bash"
>apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-monitoring-config
namespace: openshift-monitoring
data:
config.yaml: |
alertmanagerMain:
volumeClaimTemplate:
spec:
storageClassName: local-storage
volumeMode: Filesystem
resources:
requests:
storage: 20Gi
</code></pre>
<pre><code class="bash"
>$ oc exec -it prometheus-k8s-0 -c prometheus -n openshift-monitoring -- ls -l /prometheus
$ oc exec -it prometheus-k8s-0 -c prometheus -n openshift-monitoring -- df -h /prometheus
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-22566924184226655832022-11-02T11:27:00.002+01:002022-11-02T11:48:51.557+01:00OpenShift 4.6 Automation and Integration: Storage<h2>Overview</h2>
<p>
3.1. Persistent storage overview<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-overview_understanding-persistent-storage" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-overview_understanding-persistent-storage</a>
</p>
<p>The OpenShift storage architecture has three primary components:</p>
<ul>
<li>Storage Classes</li>
<li>Persistent Volumes</li>
<li>Persistent Volume Claims</li>
</ul>
<h3>Persistent Volume Claims (pvc)</h3>
<p>The project defines pvc with following</p>
<ul>
<li>Storage Size: [G|Gi...]</li>
<li>Storage Class: </li>
<li>Access Mode: [ReadWriteMany|ReadWriteOnce|ReadOnlyMany]</li>
<li>Volume Mode: [Filesystem|Block|Object]</li>
</ul>
<h3>Persistent Volume (pv)</h3>
<p>
4.11. Persistent storage using NFS<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-nfs" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-nfs</a>
</p>
<p>Example Persistent Volume</p>
<pre><code class="bash"
>apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0001
spec:
capacity:
storage: 5Gi
storageClassName: nfs-storage
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
nfs:
path: /tmp
server: 172.17.0.2
persistentVolumeReclaimPolicy: Retain
</code></pre>
<p>This persistent volume uses the NFS volume plug-in. The nfs section defines parameters that the NFS volume plug-in requires to mount the volume on a node. This section includes sensitive NFS configuration information.</p>
<h2>Provisioning and Binding Persistent Volumes</h2>
<ul>
<li>Install a storage operator</li>
<li>Write and use Ansible Playbooks</li>
</ul>
<h2>Persistent Volume Reclaim Policy</h2>
<p>
3.2.6. Reclaim policy for persistent volumes<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#reclaiming_understanding-persistent-storage" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#reclaiming_understanding-persistent-storage</a>
</p>
<ul>
<li><b>Delete</b>: reclaim policy deletes both the PersistentVolume object from OpenShift Container Platform and the associated storage asset in external infrastructure, such as AWS EBS or VMware vSphere. All dynamically-provisioned persistent volumes use a Delete reclaim policy.</li>
<li><b>Retain</b>: Reclaim policy allows manual reclamation of the resource for those volume plug-ins that support it.</li>
<li><b>Recycle</b>: Reclaim policy recycles the volume back into the pool of unbound persistent volumes once it is released from its claim.</li>
</ul>
<h2>Supported access modes for PVs</h2>
<p>
Table 3.2. Supported access modes for PVs<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#pv-access-modes_understanding-persistent-storage" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#pv-access-modes_understanding-persistent-storage</a>
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOIwQk6K6SRQ9cMvSEWTnC-FiReLgSZGh0jYJjBc9gb3QyjoLBUXGEqyb7VaK8NnDip0b4LjP1gHds3y1-x75rYnbsoJeRB3ydjTsusBgmjhpyEezdZYCrud4k4Dzf6Cxw4wIGxwLDDYN9mLrJmZcrR--RjuQfip-tBW2VIArqH-Hrgk0exlaJdkm/s1374/Screenshot%20from%202022-11-01%2018-17-43.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="400" data-original-height="1374" data-original-width="847" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOIwQk6K6SRQ9cMvSEWTnC-FiReLgSZGh0jYJjBc9gb3QyjoLBUXGEqyb7VaK8NnDip0b4LjP1gHds3y1-x75rYnbsoJeRB3ydjTsusBgmjhpyEezdZYCrud4k4Dzf6Cxw4wIGxwLDDYN9mLrJmZcrR--RjuQfip-tBW2VIArqH-Hrgk0exlaJdkm/s400/Screenshot%20from%202022-11-01%2018-17-43.png"/></a></div>
<h2>Available dynamic provisioning plug-ins</h2>
<p>
7.2. Available dynamic provisioning plug-ins<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#available-plug-ins_dynamic-provisioning" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#available-plug-ins_dynamic-provisioning</a>
</p>
<h2>Setting a Default Storage Class</h2>
<p>
7.3.2. Storage class annotations<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#storage-class-annotations_dynamic-provisioning" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#storage-class-annotations_dynamic-provisioning</a>
</p>
<pre><code class="bash"
>apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: "true"
</code></pre>
<h2>Restricting Access to Storage Resources</h2>
<p>
5.1.1. Resources managed by quotas<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/applications/index#quotas-resources-managed_quotas-setting-per-project" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/applications/index#quotas-resources-managed_quotas-setting-per-project</a>
</p>
<table border="1">
<tr>
<td>requests.storage</td>
<td>The sum of storage requests across all persistent volume claims in any state cannot exceed this value.</td>
</tr>
<tr>
<td>persistentvolumeclaims</td>
<td>The total number of persistent volume claims that can exist in the project.</td>
</tr>
<tr>
<td><storage-class-name>.storageclass.storage.k8s.io/requests.storage</td>
<td>The sum of storage requests across all persistent volume claims in any state that have a matching storage class, cannot exceed this value.</td>
</tr>
<tr>
<td><storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaims</td>
<td>The total number of persistent volume claims with a matching storage class that can exist in the project.</td>
</tr>
</table>
<h2>Block Volume</h2>
<p>
3.5.1. Block volume examples<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#block-volume-examples_understanding-persistent-storage" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#block-volume-examples_understanding-persistent-storage</a>
</p>
<pre><code class="bash"
>apiVersion: v1
kind: PersistentVolume
metadata:
name: block-pv
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
volumeMode: Block 1
persistentVolumeReclaimPolicy: Retain
fc:
targetWWNs: ["50060e801049cfd1"]
lun: 0
readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: block-pvc
spec:
accessModes:
- ReadWriteOnce
volumeMode: Block
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
name: pod-with-block-volume
spec:
containers:
- name: fc-container
image: fedora:26
command: ["/bin/sh", "-c"]
args: [ "tail -f /dev/null" ]
volumeDevices:
- name: data
devicePath: /dev/xvda
volumes:
- name: data
persistentVolumeClaim:
claimName: block-pvc
</code></pre>
<h2>Persistent storage using iSCSI</h2>
<p>
4.9. Persistent storage using iSCSI<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-iscsi" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-iscsi</a>
</p>
<p>PersistentVolume object definition</p>
<pre><code class="bash"
>apiVersion: v1
kind: PersistentVolume
metadata:
name: iscsi-pv
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
storageClassName: iscsi-blk
accessModes:
- ReadWriteOnce
iscsi:
targetPortal: 10.0.0.1:3260
iqn: iqn.2016-04.test.com:storage.target00
lun: 0
initiatorName: iqn.2016-04.test.com:custom.iqn 1
fsType: ext4
readOnly: false
</code></pre>
<h2>Persistent storage using local volumes</h2>
<h3>Installing the Local Storage Operator</h3>
<p>
4.10.1. Installing the Local Storage Operator<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#local-storage-install_persistent-storage-local" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#local-storage-install_persistent-storage-local</a>
</p>
<pre><code class="bash"
>$ oc debug node/worker06 -- lsblk
...
vdb 252:16 0 20G 0 disk
$ oc adm new-project openshift-local-storage
$ OC_VERSION=$(oc version -o yaml | grep openshiftVersion | \
grep -o '[0-9]*[.][0-9]*' | head -1)
</code></pre>
<pre><code class="bash"
>apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
name: local-operator-group
namespace: openshift-local-storage
spec:
targetNamespaces:
- openshift-local-storage
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: local-storage-operator
namespace: openshift-local-storage
spec:
channel: "${OC_VERSION}"
installPlanApproval: Automatic 1
name: local-storage-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
</code></pre>
<pre><code class="bash"
>$ oc apply -f openshift-local-storage.yaml
</code></pre>
<h3>Verify installation</h3>
<pre><code class="bash"
>$ oc -n openshift-local-storage get pods
$ oc get csv -n openshift-local-storage
NAME DISPLAY VERSION REPLACES PHASE
local-storage-operator.4.2.26-202003230335 Local Storage 4.2.26-202003230335 Succeeded
</code></pre>
<h3>Provisioning local volumes by using the Local Storage Operator</h3>
<pre><code class="bash"
>$ export CSV_NAME=$(oc get csv -n openshift-local-storage -o name)
$ oc get ${CSV_NAME} -o jsonpath='{.spec.customresourcedefinitions.owned[*].kind}{"\n"}'
LocalVolume LocalVolumeSet LocalVolumeDiscovery LocalVolumeDiscoveryResult
$ oc get ${CSV_NAME} -o jsonpath='{.metadata.annotations.alm-examples}{"\n"}'
[
{
"apiVersion": "local.storage.openshift.io/v1",
"kind": "LocalVolume",
"metadata": {
"name": "example"
},
"spec": {
"storageClassDevices": [
{
"devicePaths": [
"/dev/vde",
"/dev/vdf"
],
"fsType": "ext4",
"storageClassName": "foobar",
"volumeMode": "Filesystem"
}
]
}
}
...
]
</code></pre>
<pre><code class="bash"
>apiVersion: local.storage.openshift.io/v1
kind: LocalVolume
metadata:
name: local-storage
spec:
storageClassDevices:
- devicePaths:
- /dev/vdb
fsType: ext4
storageClassName: local-blk
volumeMode: Filesystem
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-20850965077916308892022-11-02T10:38:00.000+01:002022-11-02T10:38:07.138+01:00OpenShift 4.6 Automation and Integration: Machine Config Pool and Machine Config<h3>Introduction</h3>
<p>
1.4. About Red Hat Enterprise Linux CoreOS (RHCOS) and Ignition<br/>
1.2. About the control plane<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/architecture/index#coreos-and-ignition" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/architecture/index#coreos-and-ignition</a>
</p>
<p>Red Hat discourages directly manipulating a RHCOS configuration. Instead, provide initial instance configuration in the form of Ignition files.</p>
<p>After the instance is provisioned, changes to RHCOS are managed by the Machine Config Operator.</p>
<p>
7.2.2. Creating a machine set<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#machineset-creating_creating-infrastructure-machinesets" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#machineset-creating_creating-infrastructure-machinesets</a>
</p>
<p>
4.2.7. Customization<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/security_and_compliance/index#customization-2" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/security_and_compliance/index#customization-2</a>
</p>
<h3>Example MachineConfig (mc)</h3>
<pre><code class="bash"
>apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: infra
name: 50-foo-config
spec:
config:
ignition:
version: 3.1.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,LS0t...LQo=
filesystem: root
mode: 0644
path: /etc/foo-config
</code></pre>
<p>
7.2.4. Creating a machine config pool for infrastructure machines<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#creating-infra-machines_creating-infrastructure-machinesets" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#creating-infra-machines_creating-infrastructure-machinesets</a>
</p>
<h3>Example MachineConfigPool (mcp)</h3>
<pre><code class="bash"
>apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
name: infra
spec:
machineConfigSelector:
matchExpressions:
- {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,infra]}
nodeSelector:
matchLabels:
node-role.kubernetes.io/infra: ""
</code></pre>
<pre><code class="bash"
>$ oc get mcp
$ oc get mc --show-labels
$ oc get mc --selector=machineconfiguration.openshift.io/role=infra
</code></pre>
<h3>Label Nodes</h3>
<p>Add a label to worker node</p>
<pre><code class="bash"
>$ oc label node/worker03 node-role.kubernetes.io/infra=
</code></pre>
<p>Remove label from worker node</p>
<pre><code class="bash"
>$ oc label node/worker03 node-role.kubernetes.io/infra-
</code></pre>
<h3>Configuring Pod Scheduling</h3>
<p>
7.4. Moving resources to infrastructure machine sets<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#moving-resources-to-infrastructure-machinesets" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#moving-resources-to-infrastructure-machinesets</a>
</p>
<p>
3.7. Placing pods on specific nodes using node selectors<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-node-selectors" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index#nodes-scheduler-node-selectors</a>
</p>
<pre><code class="bash"
>apiVersion: apps/v1
kind: Deployment
metadata:
name: foo
spec:
replicas: 2
selector:
matchLabels:
app: foo
template:
metadata:
labels:
app: foo
spec:
nodeSelector:
node-role.kubernetes.io/infra: ""
containers:
...
</code></pre>
<p>
4.1.2. Creating daemonsets<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/nodes/index</a>
</p>
<p>If you fail to debug a node, this could be because of a defaultNodeSelector is defined, then you must specify a node selector to override the default.</p>
<pre><code class="bash"
>$ oc adm new-project debug --node-selector=""
$ oc debug node/master03 -n debug
</code></pre>
<h3>Observing Machine Config Pool Updates</h3>
<p><a href="https://github.com/openshift/machine-config-operator/blob/master/docs/MachineConfigController.md" target="_blank">https://github.com/openshift/machine-config-operator/blob/master/docs/MachineConfigController.md</a></p>
<p>Following annotations on node object will be used by UpdateController to coordinate node update with MachineConfigDaemon.</p>
<ul>
<li><b>machine-config-daemon.v1.openshift.com/currentConfig</b>: defines the current MachineConfig applied by MachineConfigDaemon.</li>
<li><b>machine-config-daemon.v1.openshift.com/desiredConfig</b>: defines the desired MachineConfig that need to be applied by MachineConfigDaemon</li>
<li><b>machine-config-daemon.v1.openshift.com/state</b>: defines the state of the MachineConfigDaemon, It can be done, working and degraded.</li>
</ul>
<pre><code class="bash"
>$ oc describe node/worker03
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-53785419413897732832022-11-02T10:28:00.001+01:002022-11-02T10:28:31.889+01:00OpenShift 4.6 Automation and Integration: Adding Working Nodes<h2>Installer-Provisioned Infrastructure</h2>
<p>
3.2. Scaling a machine set manually<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#machineset-manually-scaling_manually-scaling-machineset" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#machineset-manually-scaling_manually-scaling-machineset</a>
</p>
<p>
In installer-provisioned OCP cluster does the the Machine API automatically performs scaling operations, just modify the number of replicas specified in a Machine Set, and the OCP communicates to the provider to provision or deprovision instances.
</p>
<h2>User-Provisioned Infrastructure</h2>
<h3>Adding compute machines to bare metal</h3>
<p>
10.4. Adding compute machines to bare metal<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#adding-bare-metal-compute-user-infra" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#adding-bare-metal-compute-user-infra</a>
</p>
<p>Here you must create the new machines yourself. You can create new Red Hat Enterprise Linux CoreOS (RHCOS) machines either from ISO image or use Preboot eXecution Environment (PXE) boot.</p>
<p>PXE relies on a set of very basic technologies:</p>
<ul>
<li>Dynamic Host Configuration Protocol (DHCP) for locating instances.</li>
<li>Trivial File Transfer Protocol (TFTP) for serving the PXE files.</li>
<li>HTTP for the ISO images and configuration files.</li>
</ul>
<p>Example PXE. <b>NOTE</b> THE APPEND PARAMETERS NEED TO BE ON A SINGLE LINE</p>
<pre><code class="bash"
>DEFAULT pxeboot
TIMEOUT 20
PROMPT 0
LABEL pxeboot
KERNEL http://<HTTP_server>/rhcos-<version>-live-kernel-<architecture>
APPEND initrd=http://<HTTP_server>/rhcos-<version>-live-initramfs.<architecture>.img
coreos.inst.install_dev=/dev/sda
coreos.inst.ignition_url=http://<HTTP_server>/worker.ign
coreos.live.rootfs_url=http://<HTTP_server>/rhcos-<version>-live-rootfs.<architecture>.img
coreos.inst=yes
console=tty0
console=ttyS0
ip=dhcp rd.neednet=1
</code></pre>
<p>The <b>coreos.inst.ignition_url</b> param points to a working ignition file.</p>
<p>
5.1.10. Creating the Kubernetes manifest and Ignition config files<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/installing/index#installation-user-infra-generate-k8s-manifest-ignition_installing-bare-metal" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/installing/index#installation-user-infra-generate-k8s-manifest-ignition_installing-bare-metal</a>
</p>
<p>The OpenShift Container Platform installation program ($ ./openshift-install create manifests --dir <installation_directory>) generates</p>
<ul>
<li>bootstrap.ign</li>
<li>master.ign</li>
<li>worker.ign</li>
</ul>
<p>Example worker.ign</p>
<pre><code class="bash"
>{
"ignition": {
"config": {
"merge": [
{
"source": "https://api-int.mkk.example.com:22623/config/worker",
"verification": {}
}
]
},
"security": {
"tls": {
"certificateAuthorities": [
{
"source": "data:text/plain;charset=utf-8;base64,XXX...XX",
"verification": {}
}
]
}
},
"version": "3.1.0"
},
}
</code></pre>
<p><b>certificateAuthorities</b> contains the custom truststore for the internal CA. You can check a HTTPS endpoint cert chain with openssl, and for above endpoint.</p>
<pre><code class="bash"
>$ openssl s_client -connect api-int.mkk.example.com:22623 -showcerts
</code></pre>
<p>And you can check that it is the same Root CA in worker.ign with</p>
<pre><code class="bash"
>$ echo "XXX...XX" | base64 -d | openssl -text -noout
</code></pre>
<h3>Red Hat OpenStack Platform HAProxy</h3>
<p>
Chapter 5. Using HAProxy<br/>
<a href="https://access.redhat.com/documentation/fr-fr/red_hat_openstack_platform/10/html-single/understanding_red_hat_openstack_platform_high_availability/index#haproxy" target="_blank">https://access.redhat.com/documentation/fr-fr/red_hat_openstack_platform/10/html-single/understanding_red_hat_openstack_platform_high_availability/index#haproxy</a>
</p>
<p>On a Red Hat OpenStack Platform you must then update the HAProxy (/etc/haproxy/haproxy.cfg) with the nodes</p>
<h3>Approving the certificate signing requests for your machines</h3>
<p>
10.4.3. Approving the certificate signing requests for your machines<br/>
<a href="https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#installation-approve-csrs_adding-bare-metal-compute-user-infra" target="_blank">https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/machine_management/index#installation-approve-csrs_adding-bare-metal-compute-user-infra</a>
</p>
<pre><code class="bash"
>$ oc get csr -A
$ oc adm certificate approve csr-abc
</code></pre>
<h3>Verify</h3>
<p>You should now see the new worker nodes, but it will take some time for them to reach Ready state.</p>
<pre><code class="bash"
>$ oc get nodes
</code></pre>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-39666108280036591152022-10-29T16:25:00.002+02:002022-10-29T16:25:55.939+02:00RH Satellite 6.11: Registering Hosts to Satellite<h2>Fix DNS in Test Environment</h2>
<p>Manually set DNS on every servers</p>
<pre><code class="bash"
># cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.22 satellite.mkk.se satellite
192.168.122.33 rhel90-01.mkk.se rhel90-01
</code></pre>
<h2>Prepare Hosts for Satellite Registration</h2>
<pre><code class="bash"
># timedatectl
Local time: Sat 2022-07-02 00:46:59 UTC
Universal time: Sat 2022-07-02 00:46:59 UTC
RTC time: Sat 2022-07-02 00:46:59
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
# chronyc -n tracking
</code></pre>
<h2>Registering Hosts to Satellite</h2>
<p>
Chapter 3. Registering Hosts to Satellite<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_hosts/index#Registering_Hosts_to_Server_managing-hosts" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_hosts/index#Registering_Hosts_to_Server_managing-hosts</a>
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLPxaga0TrqozkaHLydSvmVktXrBXA0dcgjbufcIvagwfbRbrlQcAzP-4jUCohVmnONu2_17LXQtqC0CUAlRY6PewpHWKuH99iztZRIpKGvYTftRsa0KuNquNYs6C-WcCg-Q9SzKbQ9klF-TFnDZOI88yBm5bSBQKG6gfD68T1PY1hQS_2OJuVvY3b/s796/Screenshot%20from%202022-10-29%2015-51-29.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="731" data-original-width="796" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLPxaga0TrqozkaHLydSvmVktXrBXA0dcgjbufcIvagwfbRbrlQcAzP-4jUCohVmnONu2_17LXQtqC0CUAlRY6PewpHWKuH99iztZRIpKGvYTftRsa0KuNquNYs6C-WcCg-Q9SzKbQ9klF-TFnDZOI88yBm5bSBQKG6gfD68T1PY1hQS_2OJuVvY3b/s400/Screenshot%20from%202022-10-29%2015-51-29.png"/></a></div>
<p>Clear any previous subscription and registration.</p>
<pre><code class="bash"
>[root@rhel90-01 ~]# subscription-manager remove --all
[root@rhel90-01 ~]# subscription-manager unregister
[root@rhel90-01 ~]# subscription-manager clean
</code></pre>
<p>Download and install the consumer RPM.</p>
<pre><code class="bash"
>[root@rhel90-01 tmp]# yum localinstall http://satellite.mkk.se/pub/katello-ca-consumer-latest.noarch.rpm
</code></pre>
<p>Register the host with a Satellite admin account</p>
<pre><code class="bash"
># subscription-manager register --org <Organization> --environment <Lifecycle Environment Name>/<Content View>
</code></pre>
<pre><code class="bash"
>[root@rhel90-01 tmp]# subscription-manager register --org MKK --environment Development/Base
Registering to: satellite.mkk.se:443/rhsm
Username: admin
Password:
The system has been registered with ID: a2bdbfbb-13f3-430f-9a11-431da3a43b87
The registered system name is: rhel90-01.mkk.se
[root@rhel90-01 tmp]# subscription-manager status
+-------------------------------------------+
System Status Details
+-------------------------------------------+
Overall Status: Disabled
Content Access Mode is set to Simple Content Access. This host has access to content, regardless of subscription status.
System Purpose Status: Disabled
[root@rhel90-01 tmp]# subscription-manager repos --list
+----------------------------------------------------------+
Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID: rhel-9-for-x86_64-baseos-rpms
Repo Name: Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)
Repo URL: https://satellite.mkk.se/pulp/content/MKK/Development/Base/content/dist/rhel9/$releasever/x86_64/baseos/os
Enabled: 1
Repo ID: rhel-9-for-x86_64-appstream-rpms
Repo Name: Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)
Repo URL: https://satellite.mkk.se/pulp/content/MKK/Development/Base/content/dist/rhel9/$releasever/x86_64/appstream/os
Enabled: 1
[root@rhel90-01 tmp]# cat /etc/yum.repos.d/redhat.repo
#
# Certificate-Based Repositories
# Managed by (rhsm) subscription-manager
#
# *** This file is auto-generated. Changes made here will be over-written. ***
# *** Use "subscription-manager repo-override --help" if you wish to make changes. ***
#
# If this file is empty and this system is subscribed consider
# a "yum repolist" to refresh available repos
#
[rhel-9-for-x86_64-appstream-rpms]
name = Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)
baseurl = https://satellite.mkk.se/pulp/content/MKK/Development/Base/content/dist/rhel9/$releasever/x86_64/appstream/os
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
sslverify = 1
sslcacert = /etc/rhsm/ca/katello-server-ca.pem
sslclientkey = /etc/pki/entitlement/7011859186933837834-key.pem
sslclientcert = /etc/pki/entitlement/7011859186933837834.pem
metadata_expire = 1
enabled_metadata = 1
[rhel-9-for-x86_64-baseos-rpms]
name = Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)
baseurl = https://satellite.mkk.se/pulp/content/MKK/Development/Base/content/dist/rhel9/$releasever/x86_64/baseos/os
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
sslverify = 1
sslcacert = /etc/rhsm/ca/katello-server-ca.pem
sslclientkey = /etc/pki/entitlement/7011859186933837834-key.pem
sslclientcert = /etc/pki/entitlement/7011859186933837834.pem
metadata_expire = 1
enabled_metadata = 1
</code></pre>
<p>Verify registred host on Satellite</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Hosts -> Content Hosts</b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNAO9EATzHu-EEfwZPT4J7O1k9Fq2pHsZhBl9Y_5efPW36o4vN94KjC0VDRgulT-dJETi__DmQ1gfh8PUFzlit3pwe_4r5wWPSb4-z6t715dw6hB5ESVvilQP2WFMaVHpAg0aybXNXKgKNY_914U5Bq-LpJVDpkEevluHz3Zs20dBzGdLlk1ioWl2f/s1521/Screenshot%20from%202022-10-29%2016-06-07.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="548" data-original-width="1521" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNAO9EATzHu-EEfwZPT4J7O1k9Fq2pHsZhBl9Y_5efPW36o4vN94KjC0VDRgulT-dJETi__DmQ1gfh8PUFzlit3pwe_4r5wWPSb4-z6t715dw6hB5ESVvilQP2WFMaVHpAg0aybXNXKgKNY_914U5Bq-LpJVDpkEevluHz3Zs20dBzGdLlk1ioWl2f/s320/Screenshot%20from%202022-10-29%2016-06-07.png"/></a></div>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAKO3slIdpGMDTntcR3HYaiESSNViI6rdwArgczV3ZuTrXeaetG8YUcJ6UXVLGKL8XqAFz7Nc2m4uMjxMCrz11MVcSE-W-Yw50aqAi35sMpXpdL9cuE3E5D218cEhg0heQvgxBPBucY2sNikTdUADTGwV904Ee-zkMsyAAxtn9hyQcNCv2tkqa4VfC/s1595/Screenshot%20from%202022-10-29%2016-11-15.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="1274" data-original-width="1595" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAKO3slIdpGMDTntcR3HYaiESSNViI6rdwArgczV3ZuTrXeaetG8YUcJ6UXVLGKL8XqAFz7Nc2m4uMjxMCrz11MVcSE-W-Yw50aqAi35sMpXpdL9cuE3E5D218cEhg0heQvgxBPBucY2sNikTdUADTGwV904Ee-zkMsyAAxtn9hyQcNCv2tkqa4VfC/s320/Screenshot%20from%202022-10-29%2016-11-15.png"/></a></div>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0tag:blogger.com,1999:blog-7808117296491924045.post-75694505732743730072022-10-29T10:18:00.002+02:002022-10-29T10:18:45.636+02:00RH Satellite 6.11: Managing Red Hat Subscription, Satellite Repository, Lifecycle Environment and Content View<h2>Manage Subscriptions and Content</h2>
<h3>Create a Subscription Manifest</h3>
<p>
Chapter 5. Managing Red Hat Subscriptions<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Managing_Red_Hat_Subscriptions_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Managing_Red_Hat_Subscriptions_content-management</a>
</p>
<p><b>https://access.redhat.com/ -> Subscriptions -> Subscription Allocations -> Create New subscription allocation</b></p>
<pre><code class="bash"
>Name: mkk-satellite
Type (select a type and version of the subscription management application that you are using): Satellite 6.11
</code></pre>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFJWdxJXBBkRZf922sBtdjasu1bDTzE7QgwE1GuA3UkC1jE8KVfo8EA2VfwojNMuQr0grq70OFdzXsI2W6ywvea3guVjWRYkGHjFZcJ8ZSbUhGyce46o4Mf5Iw0j-f6HgwSftqeP1S2E5tsrU5EfjtWx6RVavb9YMQxaxv4Tl9LBR0eSzw2paELMQG/s1277/Screenshot%20from%202022-10-28%2020-41-02.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="765" data-original-width="1277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFJWdxJXBBkRZf922sBtdjasu1bDTzE7QgwE1GuA3UkC1jE8KVfo8EA2VfwojNMuQr0grq70OFdzXsI2W6ywvea3guVjWRYkGHjFZcJ8ZSbUhGyce46o4Mf5Iw0j-f6HgwSftqeP1S2E5tsrU5EfjtWx6RVavb9YMQxaxv4Tl9LBR0eSzw2paELMQG/s320/Screenshot%20from%202022-10-28%2020-41-02.png"/></a></div>
<p><b>Subscription Allocations -> mkk-satellite -> Subscriptions -> Add Subscriptions</b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOL_322vJgTZdpfZZM7rkeaSJj6GzykVPBzlFi_LIobW85fNMvQkGceYoCIxsipO58V1QG5lrDEihdNsDMrFy0-eM1TpUdY9_bBTFLEYGDW2mebgt8s6iNi-ZxHKEr_N3nm519sid5RtGpELj7hND8fsHFjWsx7DB1r5KQ24tnFLLZR0BzrcJ-NjX6/s1278/Screenshot%20from%202022-10-28%2020-44-30.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="779" data-original-width="1278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOL_322vJgTZdpfZZM7rkeaSJj6GzykVPBzlFi_LIobW85fNMvQkGceYoCIxsipO58V1QG5lrDEihdNsDMrFy0-eM1TpUdY9_bBTFLEYGDW2mebgt8s6iNi-ZxHKEr_N3nm519sid5RtGpELj7hND8fsHFjWsx7DB1r5KQ24tnFLLZR0BzrcJ-NjX6/s320/Screenshot%20from%202022-10-28%2020-44-30.png"/></a></div>
<p>You enable <a href="https://access.redhat.com/articles/simple-content-access" target="_blank">Simple Content Access (SCA)</a> separately for each organization, allowing you to maintain some organizations with the SCA behavior and others without the behavior.</p>
<h3>Export a Subscription Manifest from the Customer Portal</h3>
<p><b>Subscription Allocations -> mkk-satellite -> Subscriptions -> Export Manifest</b></p>
<h3>Import a Subscription Manifest to Satellite Server</h3>
<p>
3.9. Importing a Red Hat Subscription Manifest into Satellite Server<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_connected_network_environment/index#Importing_a_Red_Hat_Subscription_Manifest_into_Server_satellite" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_connected_network_environment/index#Importing_a_Red_Hat_Subscription_Manifest_into_Server_satellite</a>
</p>
<p>Select Organization to associate the subscription manifest to</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Content -> Subscriptions -> Manage Manifest</b></p>
<pre><code class="bash"
>CDN Configuration
CDN Configuration for Red Hat Content
Red Hat CDN
URL: https://cdn.redhat.com
Manifest
Subscription Manifest
Import New Manifest
</code></pre>
<h3>Verify Organization, Location and theirs Association</h3>
<pre><code class="bash"
>[root@satellite ~]# hammer organization list
---|-------|------|-------------|------
ID | TITLE | NAME | DESCRIPTION | LABEL
---|-------|------|-------------|------
1 | MKK | MKK | | MKK
---|-------|------|-------------|------
[root@satellite ~]# hammer location list
---|-----------|-----------|------------
ID | TITLE | NAME | DESCRIPTION
---|-----------|-----------|------------
2 | Stockholm | Stockholm |
---|-----------|-----------|------------
</code></pre>
<p>Verify that Location is associated with Organization</p>
<pre><code class="bash"
>[root@satellite ~]# hammer location list --organization MKK
---|-----------|-----------|------------
ID | TITLE | NAME | DESCRIPTION
---|-----------|-----------|------------
2 | Stockholm | Stockholm |
---|-----------|-----------|------------
</code></pre>
<h2>Synchronize Red Hat Content</h2>
<h3>To Enable a Repository in an Organization</h3>
<p>
6.5. Enabling Red Hat Repositories<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Enabling_Red_Hat_Repositories_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Enabling_Red_Hat_Repositories_content-management</a>
</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Content -> Red Hat Repositories -> <Repository Name></b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitHQfXL4JdDEb23EsTTF6oK6qGK3-YM9PMMmV4-IkA9xqhfPF_4jUeCDug70qkjg8MXBrRc7f7wVq79QKzFh-k0HxtbIaGJzmM_w3qbhitF2oUxgtUwVAHuyxLzyBW9XX0OA0kC_-UUWq9e4q_hMHyaLNw4PGgHEoKnqIOvRWni0ryDi92F2-isng1/s2560/Screenshot%20from%202022-10-29%2007-57-03.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="1208" data-original-width="2560" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitHQfXL4JdDEb23EsTTF6oK6qGK3-YM9PMMmV4-IkA9xqhfPF_4jUeCDug70qkjg8MXBrRc7f7wVq79QKzFh-k0HxtbIaGJzmM_w3qbhitF2oUxgtUwVAHuyxLzyBW9XX0OA0kC_-UUWq9e4q_hMHyaLNw4PGgHEoKnqIOvRWni0ryDi92F2-isng1/s320/Screenshot%20from%202022-10-29%2007-57-03.png"/></a></div>
<h3>To Manually Synchronize Red Hat Product Repositories</h3>
<p>
6.6. Synchronizing Repositories<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Synchronizing_Repositories_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Synchronizing_Repositories_content-management</a>
</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Content -> Products -> <Product Name> -> <Repository Name> -> Sync Now</b></p>
<h3>To Schedule Synchronization of Red Hat Product Repositories</h3>
<p>Create Sync Plan</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Content -> Sync Plans -> Create Sync Plan</b></p>
<pre><code class="bash"
>Name: rhel8
Interval: hourly
Start Date: Today
Start Time: Now
</code></pre>
<p><b>Products -> Add -> <Select Product Name> -> Add Selected</b></p>
<h3>Define Download Policies</h3>
<p>
6.7. Download Policies Overview<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Download_Policies_Overview_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Download_Policies_Overview_content-management</a>
</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Administer -> Settings -> Content</b></p>
<pre><code class="bash"
>Default Custom Repository download policy: immediate
Default Red Hat Repository download policy: on_demand
</code></pre>
<h3>Verify Content from the CLI</h3>
<pre><code class="bash"
>[root@satellite ~]# hammer repository list
---|----------------------------------------------------------|-------------------------------------|--------------|----------------------------------------------------------------
ID | NAME | PRODUCT | CONTENT TYPE | URL
---|----------------------------------------------------------|-------------------------------------|--------------|----------------------------------------------------------------
3 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8 | Red Hat Enterprise Linux for x86_64 | yum | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os
1 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8 | Red Hat Enterprise Linux for x86_64 | yum | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os
---|----------------------------------------------------------|-------------------------------------|--------------|----------------------------------------------------------------
[root@satellite ~]# hammer repository synchronize --id 1
</code></pre>
<h3>Verify Content from the Web UI</h3>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> Monitor -> Recurring Logics</b></p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Products -> <Product Name> -> <Repository Name></b></p>
<h2>Create Software Lifecycles</h2>
<p>
7.3. Creating a Life Cycle Environment Path<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Creating_a_Life_Cycle_Environment_Path_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Creating_a_Life_Cycle_Environment_Path_content-management</a>
</p>
<p>The first environment in every environment path is the Library environment that receives synced content from available sources. The Library environment is continuously syncing with source repositories, which are configured by the sync plans that you assigned to each repository.</p>
<p>Create a lifecycle environment path:</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Lifecycle Environments -> Create Environment Path</b></p>
<pre><code class="bash"
>Name: Development
Label: Development
Descriptiom: Development environment
Add New Environment
Name: Production
Label: Production
Descriptiom: Production environment
Prior Environment: Development
</code></pre>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsnPUis3ZYest0JG8SBT7HvkMZShtS2gV9tn2o3ZBmo8SyoPy2vUDkqbdPG-AyLSX42iRk0TRcMbRZ5-EHs7EUPhT-yJtiEqpMbEqdB3HsGV3yxlxNN0VNIaXW1HvVq4efaWWd8WCkyjk2KEBdadu9o-H1iaF1nkBpLr8HwrbKImsuZvDPF3nMK5JC/s1275/Screenshot%20from%202022-10-29%2008-49-28.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="618" data-original-width="1275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsnPUis3ZYest0JG8SBT7HvkMZShtS2gV9tn2o3ZBmo8SyoPy2vUDkqbdPG-AyLSX42iRk0TRcMbRZ5-EHs7EUPhT-yJtiEqpMbEqdB3HsGV3yxlxNN0VNIaXW1HvVq4efaWWd8WCkyjk2KEBdadu9o-H1iaF1nkBpLr8HwrbKImsuZvDPF3nMK5JC/s320/Screenshot%20from%202022-10-29%2008-49-28.png"/></a></div>
<h3>Verify Content from the CLI</h3>
<pre><code class="bash"
>[root@satellite ~]# hammer lifecycle-environment list --organization MKK
---|-------------|------------
ID | NAME | PRIOR
---|-------------|------------
2 | Development | Library
1 | Library |
3 | Production | Development
---|-------------|------------
[root@satellite ~]# hammer lifecycle-environment paths --organization MKK
------------------------------------
LIFECYCLE PATH
------------------------------------
Library >> Development >> Production
------------------------------------
</code></pre>
<h2>Publish and Promote Content Views</h2>
<p>
Chapter 8. Managing Content Views<br/>
<a href="https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Managing_Content_Views_content-management" target="_blank">https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_content/index#Managing_Content_Views_content-management</a>
</p>
<h3>Content Views</h3>
<p>A content view is a customized content repository to define the software that a specific environment uses.</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Content Views -> Create New View</b></p>
<pre><code class="bash"
>Name: Base
Label: Base
Description: Base packages
</code></pre>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjW4T9C6wvsbNHehuFzYeFq8QmNTmg7NT91Z0dXQ7S9CEvJseGeWlkncoE32JzEZspdIzHyQ36BkZj75iWMwuM0pnS0sH32uzWF5HMREQi32D1cR0Fz7O7qfGA37HrMrE5GrynmVAtfGhGlzpdZyMTSXhNygV40eLl8_kmZzdd_gi0lDitD-__uJe-D/s1278/Screenshot%20from%202022-10-29%2009-03-17.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="1085" data-original-width="1278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjW4T9C6wvsbNHehuFzYeFq8QmNTmg7NT91Z0dXQ7S9CEvJseGeWlkncoE32JzEZspdIzHyQ36BkZj75iWMwuM0pnS0sH32uzWF5HMREQi32D1cR0Fz7O7qfGA37HrMrE5GrynmVAtfGhGlzpdZyMTSXhNygV40eLl8_kmZzdd_gi0lDitD-__uJe-D/s320/Screenshot%20from%202022-10-29%2009-03-17.png"/></a></div>
<h3>Manage Repositories in Content Views</h3>
<p>Add a repository to a content view:</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Content Views -> <Content View Name> -> Repositories -> <Select Checkbox Repository> -> Add Repositories</b></p>
<h3>Content View Filters</h3>
<h3>Publish a Content View</h3>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Content Views -> <Content View Name> -> Publish New Version</b></p>
<pre><code class="bash"
>Description: Added RHEL 8 BaseOS and AppStream RPM repo
</code></pre>
<h3>Promote a Content View</h3>
<p>After publishing a content view version to the library, you can promote the content view to the first lifecycle environment in the environment path.</p>
<p>To promote a content view:</p>
<p><b>https://satellite.mkk.se/ -> <Select Organization> -> -> Content -> Content Views -> <Content View Name> -> Versions -> vertical ellipsis menu -> Promote</b></p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNaCxCwpkU9TqsiswxjyYJiL9SeL4kIXEa3mnJeHcWobu_FHwGubtmmATm4rnfg2h0cYLxl51lwZbgLERrN8ZS_cEA3yZ4-h1kvj3epso-GZ-M6aqYH7GhIkLRM0IjbYBGgXtxRxzdQlyzBSJONa0PDFJZuLiEL7esn-BhpRUPlnb1LUFLhr6ezoMB/s1275/Screenshot%20from%202022-10-29%2009-20-12.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="660" data-original-width="1275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNaCxCwpkU9TqsiswxjyYJiL9SeL4kIXEa3mnJeHcWobu_FHwGubtmmATm4rnfg2h0cYLxl51lwZbgLERrN8ZS_cEA3yZ4-h1kvj3epso-GZ-M6aqYH7GhIkLRM0IjbYBGgXtxRxzdQlyzBSJONa0PDFJZuLiEL7esn-BhpRUPlnb1LUFLhr6ezoMB/s320/Screenshot%20from%202022-10-29%2009-20-12.png"/></a></div>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRq10pY-NKoCB1IefzkV6BfbOhS7V0UxcGYDBLha3jB_rA82ZnuW34PAqxtOOPZwWR_K4OlTGSKWzQh7meZ3sIK-mjVuVnXeKm-JFlDf8p7lrWAhJQ8gXGj3NbDoFXpX78kN7htuSmDd-CqybyF-d6_FDNn0BsPkyxAhVFo1yLeCWeNPxxO9tjpxhq/s1276/Screenshot%20from%202022-10-29%2009-21-41.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="958" data-original-width="1276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRq10pY-NKoCB1IefzkV6BfbOhS7V0UxcGYDBLha3jB_rA82ZnuW34PAqxtOOPZwWR_K4OlTGSKWzQh7meZ3sIK-mjVuVnXeKm-JFlDf8p7lrWAhJQ8gXGj3NbDoFXpX78kN7htuSmDd-CqybyF-d6_FDNn0BsPkyxAhVFo1yLeCWeNPxxO9tjpxhq/s320/Screenshot%20from%202022-10-29%2009-21-41.png"/></a></div>
<pre><code class="bash"
>Description: RHEL 8 BaseOS and AppStream RPM repo
</code></pre>
<h3>Content View Scenarios</h3>
<p>In an <b>all-in-one</b> content view scenario, the content view contains all the needed content for all of your hosts.</p>
<p>In the <b>host-specific</b> content view scenario, dedicated content views exist for each host type.</p>
Magnus K Karlssonhttp://www.blogger.com/profile/00279910012518682745noreply@blogger.com0