-
-
Notifications
You must be signed in to change notification settings - Fork 878
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(serializer): add ApiProperty::uriTemplate option #5675
feat(serializer): add ApiProperty::uriTemplate option #5675
Conversation
0a4f1b4
to
6c56f0d
Compare
Hi @GregoireHebert. This is an interesting PR. We've developed a similar feature for our application & as a matter of fact, we were also thinking about proposing a PR to api-platform on this. Trying to chime in on this PR & dissect where our solutions differs from yours. Maybe we can build the best out of both solutions 😃 Our current solution is built with a custom Normalizer which decorates the built-in Hal\ItemNormalizer (we're currently only using HAL, but the concept is most probably also usable for other formats). In this Normalizer we automatically replace all array of links with a single IRI (similar as your proposal). Functionally, this method works great for us. However, we ran into an issue with sub-optimal performance: Due to the fact, that our Normalizer decorates the built-in one, too many entities are loaded from DB and normalized (basically a waste of resources, because all these data is loaded & normalized, event if they are only discarded later-on by our custom Normalizer). This is the reason why we were interested into building this into api platform natively. Some questions & points where our solutions differ:
Apologies for throwing all these points into your PR. Kind of want to avoid, we come up with a conflicting/non-compatible PR from our side. Happy to support on implementation of the points above, if they seem sensible to you. Also looping in @carlobeltrame, who was the initial developer of our applications' solution. |
May I be silly for a minute? To me this looks a lot like you sometimes have very big collections and want to lighten the output...which actually just seems like a custom ApiResource to me. Building IRIs for your collections in the provider and exposing them with the field name used in the entity shouldn't be a complicated task. I feel like I'm missing something or severely underestimating the amount of work required in your project(s) to do it "manually" that way...is that it? |
Speaking for our use-case, it's mostly about reducing the number of network requests necessary to load a tree of related data. The following response needs 3+1 network requests to load the car and all its repairs. It's an N+1 problem, so the number of requests scale linearly with the number of repairs (very bad for performance and for frontend UX). {
"_links": {
"self": {
"href": "/cars/1"
},
"repairs": [
{
"href": "/repairs/1"
},
{
"href": "/repairs/2"
},
{
"href": "/repairs/3"
}
]
},
"id": "1"
} The following response always needs exactly 2 network requests, independent of the number of repairs: {
"_links": {
"self": {
"href": "/cars/1"
},
"repairs": {
"href": "/repairs?car=/cars/1"
}
},
"id": "1"
}
I guess the overriding the fields in the provider would work for the top-level item. However, this could get complicated very fast for embedded resources. For example when the brand resource embeds its cars: {
"_links": {
"self": {
"href": "/brands/1"
},
"cars": {
"href": "/cars?brand=/brands/1"
}
},
"_embedded": {
"cars": [
{
"_links": {
"self": {
"href": "/cars/1"
},
"repairs": {
"href": "/repairs?car=/cars/1"
}
},
"id": "1"
}
]
},
"id": "1"
} The beauty of hooking into the Normalizer is that it generates consistent results, independent of whether a resource is top-level resource or a deeply nested embedded resource. |
Ok, so with embedded resources you'd have to have everything as separate ApiResources different from your entities. That's actually the way i work with APIP right now, but i understand how that could become a problem / a huge amount of work if it's not the case for you! |
I did, but for this PR, I introduced it in the ApiProperty level only.
Definitely some things to consider. Right off the bat, I am tempted to consider the |
I love the idea. This approach has many benefits:
In my opinion, we should allow enabling this option globally, and turn this feature on by default in API Platform 4 (to do so without introducing a breaking change, we can deprecate not setting explicitly the config flag in v3, and set its default value to true in v4). Instead of relying on the search filter, this feature should be tightly coupled to "subresources". Before: {
"@id": "/brands/1",
"cars": [
{"@id": "/cars/a", ...},
{"@id": "/cars/b", ...}
]
} After: {
"@id": "/brands/1",
"cars": "/brands/1/cars"
}
I don't like it either, but we already have a similar option named like that. What about Regarding possible filters, documenting them is already supported by Hydra, so I don't think we have more to do, at least for JSON-LD. Unlike Hydra, I don't think that HAL has support for URI templates. |
Thanks @dunglas for your perspective on this and for the general support of the concept.
For simple use cases, a boolean flag or a string identifying the route might be sufficient (e.g. simple properties linking to other resources). For more complex use cases (e.g. manual getters) this is not sufficient, and more information is needed (target resource + route + URI/query parameters). Latter could also be a separate feature, which basically allows overriding links with an arbitrary route. I just though there's a case to combine the 2 use-cases, as they share some similarities. Providing an example: #[ApiResource]
class Brand {
#[ApiProperty(link: new CollectionLink(
relatedEntity: Repair::class,
parameters: [
'finished' => false,
],
callbackParameters: [
'brand' => 'getBrand',
],
))]
public function getUnfinishedRepairs(): array {
$repairs = array_map(function (Car $car) {
return array_filter(
$car->repairs->getValues(),
function (Repair $repair) { return !$repair->finished; }
);
}, $this->cars->getValues());
return array_merge(...$repairs);
}
public function getBrand() {
return $this;
}
} Which then would result in the following response: {
"@id": "/brands/1",
"cars": "/brands/1/cars",
"unfinishedRepairs": "/repairs?brand=/brands/1&finished=false",
}
HAL actually has support for URI templates (following RFC6570).
We have some code on our side to create RFC6570 templates for an ApiResource. |
I did a few changes from my initial proposal. using a string to define an It will use a GetCollection Operation IRI matching the template, but this Operation has to be defined through the related Resource class. Then for the global/true/false value possibility, I think it's a bad idea in the same way as the previous comment, the current ressource should not be the source of truth for the related resource class. It guesses too much, and can allow to select matching operation that are not the desired ones. Plus it will be easier to maintain since it uses the same mechanism than the operations, will use correctly the uriVariable already declared. Now my current proposal doesn't cover one of your need that is passing default filters values during the IRI generation. |
note: last failing tests should be green when 3.1 is merged into main. |
a8c6e3f
to
fbebb25
Compare
@@ -155,8 +155,9 @@ private function joinRelations(QueryBuilder $queryBuilder, QueryNameGeneratorInt | |||
} | |||
|
|||
$fetchEager = $propertyMetadata->getFetchEager(); | |||
$uriTemplate = $propertyMetadata->geturiTemplate(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$uriTemplate = $propertyMetadata->geturiTemplate(); | |
$uriTemplate = $propertyMetadata->getUriTemplate(); |
composer.json
Outdated
@@ -75,6 +75,7 @@ | |||
"symfony/maker-bundle": "^1.24", | |||
"symfony/mercure-bundle": "*", | |||
"symfony/messenger": "^6.1", | |||
"symfony/monolog-bundle": "^3.8", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"symfony/monolog-bundle": "^3.8", |
Use LoggerInterface from PSR
@@ -638,6 +639,13 @@ protected function getAttributeValue(object $object, string $attribute, string $ | |||
$childContext = $this->createChildContext($context, $attribute, $format); | |||
unset($childContext['iri'], $childContext['uri_variables'], $childContext['resource_class'], $childContext['operation']); | |||
|
|||
if (null !== $itemUriTemplate = $propertyMetadata->getUriTemplate()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (null !== $itemUriTemplate = $propertyMetadata->getUriTemplate()) { | |
if ($itemUriTemplate = $propertyMetadata->getUriTemplate()) { |
@@ -638,6 +639,13 @@ protected function getAttributeValue(object $object, string $attribute, string $ | |||
$childContext = $this->createChildContext($context, $attribute, $format); | |||
unset($childContext['iri'], $childContext['uri_variables'], $childContext['resource_class'], $childContext['operation']); | |||
|
|||
if (null !== $itemUriTemplate = $propertyMetadata->getUriTemplate()) { | |||
$operation = $this->resourceMetadataCollectionFactory->create($resourceClass)->getOperation($itemUriTemplate, true, true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$operation = $this->resourceMetadataCollectionFactory->create($resourceClass)->getOperation($itemUriTemplate, true, true); | |
$operation = $this->resourceMetadataCollectionFactory->create($resourceClass)->getOperation($itemUriTemplate, true, true); |
you can use named arguments for clarity on these booleans?
ffb7c8f
to
847ee8b
Compare
Thanks for progressing on this. I tried to test with our application, but noticed this doesn't work with HAL format. The code fails on Hal/Serializer/ItemNormalizer.php#L230 with a Might be an issue for other formats as well. I don't know JsonAPI enough, but the code in JsonApi/Serializer/ItemNormalizer looks very similar. |
@@ -638,6 +639,17 @@ protected function getAttributeValue(object $object, string $attribute, string $ | |||
$childContext = $this->createChildContext($context, $attribute, $format); | |||
unset($childContext['iri'], $childContext['uri_variables'], $childContext['resource_class'], $childContext['operation']); | |||
|
|||
if ($itemUriTemplate = $propertyMetadata->getUriTemplate()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would actually be cool if we can avoid populating $attributeValue
before checking for uriTemplate. This can save quite some DB requests, as the actual data is not needed to generate the uri.
I made a proposal here: GregoireHebert#1
This also includes enabling uriTemplate for non-collections. While we both have mainly collections in mind as the main purpose of this feature, I think there's no harm to also allow overriding the uriTemplate for *:1 relations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am trying to take this into account, and it gives me insomnia and headaches...
I will eventually get there, please bear with me :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤣 got it. Let me know if I can be of any help?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am getting somewhere... I think.
If you could check on your project that I dont break anything 😬
I had to change the HAL json schema that didn't allowed having object embedded (only array of objects)
But I wasn't 100% sure it was alright according to the specification.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it is ok, I'll write/update the documentation guide
@usu thanks for the feedbacks, I'll try to have a look and fix the other formats during this week or the next. |
5c90aae
to
e3bf432
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checked on my project in HAL format and everything works as intended 🚀
Thanks a lot!
|
||
if (false === $fetchEager) { | ||
if (false === $fetchEager || null !== $uriTemplate) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When items are embedded, they should still be fetched.
Should we change as following?
if (false === $fetchEager || (null !== $uriTemplate && !$propertyMetadata->isReadableLink() )) {
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try this
continue; | ||
} | ||
|
||
$childContext = $this->createChildContext($context, $relationName, $format); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What exactly is the purpose of this and the next few lines? $relation['operation']
is only populated when $relation['iri']
is populated as well, in which case the for-loop has already continued/returned above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll double check that, I may have been carried away x)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think for HAL and JSON:API formats, the behaviour is fine.
But for the JSONLD format, I feel that we should not use the uriTemplate acting as an IriOnly option, when there is one that exist for NormalizationContext. Would it make more sense that, unless we use a second ApiProperty::IriOnly
option, it only resolves the IRI with uriTemplate within the object(s) themselves ? And if the ApiProperty::IriOnly
option is true, then return the IRI string instead of an object? But then what about the IriOnly option on Normalization Context?
ping @vincentchalamon @dunglas
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As discussed on video call with Vincent and Soyuka, the behaviour is different, but doesn't interfere that much. It is acceptable.
0fdb3be
to
d7eac24
Compare
d7eac24
to
da6df8b
Compare
This feature gives control over the operation used for *toOne and *toMany relations IRI generation. When defined, API Platform will use the operation declared on the related resource that matches the uriTemplate string. In addition, this will override the value returned to be the IRI string only, not an object in JSONLD formats. For HAL and JSON:API format, the IRI will be used in links properties, and in the objects embedded or in relationship properties.
76c6942
to
8550276
Compare
8550276
to
2aa93a8
Compare
Merging this as experimental, future updates can be provided as bug fix on 3.2 (currently main). Thanks @usu @GregoireHebert ! |
Thank you very much @GregoireHebert and @usu. It even documents the property as an iri-reference string instead of an array of iri-reference strings, very cool. |
@@ -155,8 +155,9 @@ private function joinRelations(QueryBuilder $queryBuilder, QueryNameGeneratorInt | |||
} | |||
|
|||
$fetchEager = $propertyMetadata->getFetchEager(); | |||
$uriTemplate = $propertyMetadata->getUriTemplate(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@GregoireHebert This causes an issue here.
Error: Typed property ApiPlatform\Metadata\ApiProperty::$uriTemplate must not be accessed before initialization
#32 /vendor/api-platform/core/src/Metadata/ApiProperty.php(477): ApiPlatform\Metadata\ApiProperty::getUriTemplate
#31 /vendor/api-platform/core/src/Doctrine/Orm/Extension/EagerLoadingExtension.php(158): ApiPlatform\Doctrine\Orm\Extension\EagerLoadingExtension::joinRelations
#30 /vendor/api-platform/core/src/Doctrine/Orm/Extension/EagerLoadingExtension.php(98): ApiPlatform\Doctrine\Orm\Extension\EagerLoadingExtension::apply
#29 /vendor/api-platform/core/src/Doctrine/Orm/Extension/EagerLoadingExtension.php(61): ApiPlatform\Doctrine\Orm\Extension\EagerLoadingExtension::applyToItem
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
weird, it's initialized to null
inside the constructor. Can you provide a reproducer an open a new issue please?
Is it still possible to use filter params on an existing route as suggested here by @usu ? {
"@id": "/brands/1",
"cars": "/brands/1/cars",
"unfinishedRepairs": "/repairs?brand=/brands/1&finished=false",
} We can only find examples where a new route needs to be created and the id of the related resource is embedded in the route (e.g. /repairs/brand/{id}) but we'd love to just have our /repairs endpoint with the filter ?brand=brand/1 as in this example. Otherwise we'd need a new route foreach relation.... If we use the existing get route, e.g. /repairs, the route can not be found. |
@Cruiser13 Not yet AFAIK. |
Why is your route not containing these parameters as URI Variables? I think it'd be possible to at least have Not sure about the |
The absolute form of an IRI containing a scheme along with a path and optional query and fragment segments. @soyuka I tried on my side, for now, matching an operation with path parameters works. Except if we pass query parameters to the
What would you think if in a similar approach, we could declare :
That would help check that a corresponding |
This is our current approach which almost works, tries to re-use the filter defined on $brand: class Repair: #[ApiResource(
uriTemplate: '/repairs?brand=/api/repairs/{brandUuid}',
operations: [ new GetCollection() ],
uriVariables: [
'brandUuid' => new Link(toProperty: 'brand', fromClass: Brand::class),
]
)]
//...
#[ORM\ManyToOne(inversedBy: 'repairs')]
#[ApiFilter(SearchFilter::class, strategy: 'exact')]
private ?Brand $brand = null; class Brand: #[ApiProperty(writable: false, uriTemplate: '/repairs?brand=/api/brands/{brandUuid}')]
#[Link(toProperty: 'brand')]
#[ORM\OneToMany(mappedBy: 'brand', targetEntity: Repair::class)]
private Collection $repairs; This has the following downsides:
Our api identifier look like /api/brands/uuid |
@Cruiser13 I don't understand why you want this to be a query parameter and not a path parameter. @GregoireHebert This could work indeed. Not sure what the use cases are and if we're not opening pandora's box with that haha. |
Lets set the situation where you already have a collection that can serve the partial collection through a filter. There is no business value / need to create a new endpoint solely for this purpose. In the end, you would have many endpoints that pollute your API landscape. |
@soyuka the situation that @GregoireHebert describes is exactly our issue here. The filter on the GetCollection returns the same results as we'd do with an additional endpoint: But it'd be an additional endpoint with additional security check, validations and a lot more complications. |
This feature gives control over the operation used for *toOne and
*toMany relations IRI generation.
When defined, API Platform will use the operation declared on the related
resource that matches the uriTemplate string.
In addition, this will override the value returned to be the IRI string
only, not an object in JSONLD formats. For HAL and JSON:API format, the
IRI will be used in links properties, and in the objects embedded or in
relationship properties.
learn more with this guide : https://github.com/GregoireHebert/core/blob/iri-only-collection-property/docs/guides/return-the-iri-of-your-resources-relations.php