Skip to content

KEYCLOAK-15454 User Event SPI prototype - Phase 1 - #7466

Closed
vmuzikar wants to merge 1 commit into
keycloak:masterfrom
vmuzikar:user-events
Closed

vmuzikar wants to merge 1 commit into
keycloak:masterfrom
vmuzikar:user-events

Conversation

@vmuzikar

@vmuzikar vmuzikar commented Oct 2, 2020

Copy link
Copy Markdown
Contributor

Based on design proposal.

@bs-matil

bs-matil commented Oct 2, 2020

Copy link
Copy Markdown
Contributor

Hi,
I'm right now try to establish a new global user profile see #7155
I feel that your approach here will have the very same problem I had. You have to build a Event logic again for each existing UserModel/Representation if you want to support all usecases of your proposal. For example the account rest spi is using its own representation. A idp review also.

May I just got confused by the sentence "User removes an authenticator" which would be the account API right?

Are you planning to support also other user representations than idm.UserRepresentation like account.UserRepresentation, and what will the format for the CloudEvent be? How does the consumer know which type he will get? Will there be a harmonization of the types within the event spi. If yes can the harmonized objects used against account and admin api?

@vmuzikar

vmuzikar commented Oct 5, 2020

Copy link
Copy Markdown
Contributor Author

@bs-matil Thank you for your feedback!

You have to build a Event logic again for each existing UserModel/Representation if you want to support all usecases of your proposal.

That's exactly the point. :) To build a new event logic tailored for user use cases.

May I just got confused by the sentence "User removes an authenticator" which would be the account API right?

Correct.

Are you planning to support also other user representations than idm.UserRepresentation like account.UserRepresentation, and what will the format for the CloudEvent be? How does the consumer know which type he will get? Will there be a harmonization of the types within the event spi. If yes can the harmonized objects used against account and admin api?

Good point with the various representations. Maybe we could introduce a UserRepresentation just for the User Event SPI.

@bs-matil

bs-matil commented Oct 5, 2020 •

Copy link
Copy Markdown
Contributor

Yes that could be an idea. But still you would than have "the next" representation.
I get the feeling in the overall architecture of keycloak there should be "one" external representation of a user which is long living (very strict versioned) while there should be attributes on top of this which are only exists on userresource/account etc.
This way this common user could be used as transfer object between different layers of keycloak and external systems without changing contracts all the time.

e.g.

{ type: "UserAccountReprensentation"
  accountField: "",
  accountField2: "",
  user:  
{
   version: "1.0"
   data: "{ attributes : [], meta: [] } "
}

I have no clear idea how this could look like. Just a very first thought on this.

@pedroigor

Copy link
Copy Markdown
Contributor

I like the API but I'm wondering if we could make it more generic.

I don't think we should care about the different representations and an event should just hold any set of key/value pairs with contextual data so that listeners can do whatever they want with it. I mean do they need to be strongly typed? Probably versioned.

I also think that the Event API you are proposing is what could help us in other areas too such as Validation (not only for users) in both User Profile, Client, etc. I mentioned that when reviewing #7155 (comment).

I'm afraid we are adding too many things that are somewhat related at their core and that may cause some headaches in the future. For instance, in User Profile, there are "events" for users, what we should do about it?

There are so many things we can do with a well designed and general-purpose Event API ...

@vmuzikar

vmuzikar commented Oct 6, 2020

Copy link
Copy Markdown
Contributor Author

@pedroigor Thanks for the feedback!

From the discussions with @stianst I believe the focus here was meant to be really narrow – an SPI tailored only for user events.

I think a loosely typed key-value object map would be an option too, although I personally don't like this solution too much as I find it unnecessarily hard for the developer to use – keys, casting, ...

I'm afraid we are adding too many things that are somewhat related at their core and that may cause some headaches in the future. For instance, in User Profile, there are "events" for users, what we should do about it?

Yes, that could be a problem. On the other hand, that's why this is a prototype, I'm ready for some refactoring. :)

@pedroigor

Copy link
Copy Markdown
Contributor

@pedroigor Thanks for the feedback!

From the discussions with @stianst I believe the focus here was meant to be really narrow – an SPI tailored only for user events.

I think a loosely typed key-value object map would be an option too, although I personally don't like this solution too much as I find it unnecessarily hard for the developer to use – keys, casting, ...

Yeah, true. But I was thinking something like this:

{
     "jti": "4d3559ec67504aaba65d40b0363faad8",
     "eventUris":[
       "urn:ietf:params:event:SCIM:create"
     ],
     "iat": 1458496404,
     "iss":"https://scim.example.com",
     "aud":[
      "https://scim.example.com/Feeds/98d52461fa5bbc879593b7754",
      "https://scim.example.com/Feeds/5d7604516b1d08641d7676ee7"
     ],
     "sub": "https://scim.example.com/Users/44f6142df96bd6ab61e7521d9",
     "urn:ietf:params:event:SCIM:create":{
       "attributes":["id","name","userName","password","emails"],
       "values":{
         "emails":[
          {"type":"work","value":"jdoe@example.com"}
         ],
         "password":"not4u2no",
         "userName":"jdoe",
         "id":"44f6142df96bd6ab61e7521d9",
         "name":{
           "givenName":"John",
           "familyName":"Doe"
         }
       }
     }
   }

That format looks simple and easy to follow/consume, so that you have the event type, the attributes that were created/modified and their values in a separate place.

So, something around that where we only need to define the event types and keep data in an easy to consume (and automatize) format.

I'm afraid we are adding too many things that are somewhat related at their core and that may cause some headaches in the future. For instance, in User Profile, there are "events" for users, what we should do about it?

Yes, that could be a problem. On the other hand, that's why this is a prototype, I'm ready for some refactoring. :)

Yeah and a nice prototype.

Another thing to consider here too is that in the future we might start supporting CDI (Quarkus) and there we already have CDI Events.

@vmuzikar

vmuzikar commented Oct 9, 2020

Copy link
Copy Markdown
Contributor Author

Yeah, true. But I was thinking something like this:

@pedroigor Yes, sure, makes sense to me. But this representation is something to be used in the external interface, e.g. CloudEvents, right? I.e. it is up to each event listener how they "translate" the representation.

Another thing to consider here too is that in the future we might start supporting CDI (Quarkus) and there we already have CDI Events.

@pedroigor I believe this will be covered in KEYCLOAK-15456. ;)

I've rebased the changes to remove conflicts.

@pedroigor

Copy link
Copy Markdown
Contributor

Yeah, true. But I was thinking something like this:

@pedroigor Yes, sure, makes sense to me. But this representation is something to be used in the external interface, e.g. CloudEvents, right? I.e. it is up to each event listener how they "translate" the representation.

It is, and the nice thing about that format is that you have the attributes that are part of the event and their values separated. So one could easily "translate" the event by initially looking at the attributes that were set as well as the type of the event.

So, in the end, what is important is to check the event type and the metadata associated with it. With the possibility to have multiple event types within the same event (if that would be useful for certain scenarios).

We would then provide a generic API on top of that format where only the event type is strongly typed, for both marshaling and unmarshalling events.

It would be the same format for both internal and external listeners.

Another thing to consider here too is that in the future we might start supporting CDI (Quarkus) and there we already have CDI Events.

@pedroigor I believe this will be covered in KEYCLOAK-15456. ;)

I've rebased the changes to remove conflicts.

}

try {
UserEventContext userEvent = UserEvents.createByAdmin(session, rep).firePreEvent();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this really be fired in the Ressource. Should not this be implemented in the UserStorageProvider or UserStorageManager?

I am just wondering because there may be use cases where creation or deletion of a user may not be triggered by an API. For example, UserStorageManager has a feature 'import validation' which deletes users from Keycloak when they are invalid or do not exist any longern in the system integrated by the federation provider.

@abstractj abstractj added area/server area/core Core server — startup, configuration, providers/SPIs, cross-cutting concerns kind/feature Categorizes a PR related to a new feature labels Nov 18, 2020
@stianst

stianst commented Jan 8, 2021

Copy link
Copy Markdown
Contributor

Haven't read the PR yet, but had a look at the design proposal again, and a few comments to that:

  • Instead of signing or using a shared secret in authz header would it not be better to simply include an "event-update-token" or something in the event, that is then included in the response? That way we can tie the request and the response without any specific needs for authz

  • For external hooks like these we'll probably want a consistent approach to how we register "external providers" though. My thinking here was that we'd have something like a well-known endpoint implemnented by a provider that would be used to tell KC what it supports, as well as configuration like for instance just send me uuid of the user, send me the full profile, or also include groups/roles, etc.. Also, we'd want a common way to authenticate an external provider (mTLS!?!) as well as some access control mechanism (service accounts)

  • What about gRPC?

  • We have also discussed some higher-level Java API for Quarkus - something more user-friendly, CDI based, detached from the internals so we can easier keep API compatiblity, etc.

This kinda suggests to me that first round here should be focused on the "low-level" SPIm, which is the equivalent of our current SPIs, then we'd need to work an a design proposal for higher-level Java APIs (do we also want JavaScript in-process in the future!?!), as well as a design proposal for external providers.

@vmuzikar

vmuzikar commented Jan 8, 2021

Copy link
Copy Markdown
Contributor Author

@stianst Thanks for your comments.

Haven't read the PR yet, but had a look at the design proposal again, and a few comments to that:

  • Instead of signing or using a shared secret in authz header would it not be better to simply include an "event-update-token" or something in the event, that is then included in the response? That way we can tie the request and the response without any specific needs for authz

Sure, but wouldn't be JWT more secure than sending the secret (that can be later used for approving/rejecting) in each event? But yes, that could be an option.

  • For external hooks like these we'll probably want a consistent approach to how we register "external providers" though. My thinking here was that we'd have something like a well-known endpoint implemnented by a provider that would be used to tell KC what it supports, as well as configuration like for instance just send me uuid of the user, send me the full profile, or also include groups/roles, etc.. Also, we'd want a common way to authenticate an external provider (mTLS!?!) as well as some access control mechanism (service accounts)

Yes, I even think we discussed it somewhere. But I think this might even deserve a separate design doc. Maybe this (registering external providers) could be a bit independent/separated from the User Event SPI.

  • What about gRPC?

That's something to investigate but I'd stick with CloudEvents as a default option for now.

  • We have also discussed some higher-level Java API for Quarkus - something more user-friendly, CDI based, detached from the internals so we can easier keep API compatiblity, etc.

Yes, we discussed this previously too. It's currently a Phase 4 of my prototype. ;)

This kinda suggests to me that first round here should be focused on the "low-level" SPIm, which is the equivalent of our current SPIs, then we'd need to work an a design proposal for higher-level Java APIs (do we also want JavaScript in-process in the future!?!), as well as a design proposal for external providers.

Sure, that's the plan. :) See KEYCLOAK-15449.

@Captain-P-Goldfish

Copy link
Copy Markdown
Contributor

This is only done for users. In order to add cloud actions for e.g. SCIM it would be necessary to have these actions also for other resources like groups.

@stianst

stianst commented Sep 30, 2021

Copy link
Copy Markdown
Contributor

@vmuzikar @velias do we want to keep this as a base for the events work, or should this PR just be closed?

@stianst stianst self-assigned this Sep 30, 2021
@vmuzikar

Copy link
Copy Markdown
Contributor Author

AFAIK the approach for improving user events experience is a little bit something else than introduced in this PR. WDYT @velias ?

@velias

velias commented Oct 1, 2021

Copy link
Copy Markdown
Contributor

@vmuzikar yep, I agree this this is something a little bit different, I'd say calling it "user event SPI" is misleading a bit, I'd call it something like 'user approval SPI' or 'user workflow SPI'.

BTW 1 - UserProfile provider has been introduced in meantime, it centralizes actions around user (creation, update), maybe this functionality should be integrated into it rather than into action classes (but this may clash with the next BTW 2 ;-)

BTW 2 - current implementation looks to be fully synchronous, blocking requests from browser, so it expects handling of approval actions is fast enough. I do not know requirements for this feature, but have you though about more asynchronous mechanism (which would be more complex to implement for sure)?

@stianst

stianst commented Oct 6, 2021

Copy link
Copy Markdown
Contributor

Closing this for now and opened a discussion on this topic here:
#8536

@stianst stianst closed this Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Core server — startup, configuration, providers/SPIs, cross-cutting concerns kind/feature Categorizes a PR related to a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants