add __new__ NS classes

[ronaldoussoren]

Original report by Georg Seifert (Bitbucket: Schriftgestalt, GitHub: Schriftgestalt).

---

I found a comment in the Foundation `__init__.py:` (https://bitbucket.org/ronaldoussoren/pyobjc/src/default/pyobjc-framework-Cocoa/Lib/Foundation/__init__.py#lines-110)

# XXX: add __new__, __getitem__ and __iter__ as well

`__new__` should be easy.

Add this method

def NSObject__new__(typ, *args, **kwargs):
  return typ.alloc().init()

And use it like this:

objc.addConvenienceForClass('NSHashTable', (
    ('__new__',      NSObject__new__)
  )
)

‌

[ronaldoussoren]

Original comment by Georg Seifert (Bitbucket: Schriftgestalt, GitHub: Schriftgestalt).

---

I just tried to implement this and it doesn't seem to work. This did:

NSHashTable.__new__ = staticmethod(GSObject__new__)

And immutable classes needs a `_init__` method to handle arguments.

[ronaldoussoren]

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren).

---

It is not as easy as this, __new__ needs to recognise keyword arguments and forward those to the right ObjC init method.

I do want to add more __new__ methods, but I’m afraid that will end up being custom methods for most classes.

There’s also classes that either cannot be created using the usual API but only through factory methods or other APIs. Those should not have __new__ (or a new that raises an appropriate error message).

‌

[ronaldoussoren]

I'm thinking about a way to do this cleanly, and not just for new. In particular, I'm thinking about a way to add PEP8-compliant "aliases" to ObjC classes using the metadata system.

Doing this will be a lot of work, but would result in Cocoa classes that are much nicer to use from Python.

The hard parts will be designing a system where subclassing still feels natural (the easiest solution is to force subclassers to use the regular "ugly" names, but that's not very nice), and devising a naming scheme for the PEP8 names (one that is predictable and can be scripted).

#285 and #198 also need changes to the metadata system.

[ronaldoussoren]

I've started looking into this and have some ideas, but also a problem.

First the basic idea (not fully fleshed out, hence vague):

• Add a __new__ to all classes that accepts keyword(-only) arguments based on the init selects in ObjC

  This would either be a generic __new__ implementation with a class attribute to set the init variants, or
  a helper that dynamically generates the __new__ implementation based on the init variants. The former is likely
  better for easily handing superclass variants, the latter is easier to adapt for other use-cases.

  E.g. behaving like (misusing typing.overload syntax):

  class MDLMaterialPropertyNode(NSObject):
     @overload
     def __new__(cos): 
         return cls.alloc().init()
  
     @overload
     def __new__(cls, *, inputs, outputs, evaluationFunction): 
         return cls.alloc().initWithInputs_outputs_evaluationFunction_(inputs, outputs, evaluationFunction)
  
     # ... more overloads for other init variants if present

• Try to keep close to the method used by Swift to translate names (AFAIK the first argument could be positional instead of kw-only)

• Possibly add a way to private defaults for the keyword arguments (which would require more manual work, but can be convenient).

• This will not be automagic based on data in the ObjC runtime, but will require support in framework bindings (with tooling to generate that support)

This seems easy enough to implement, and can later be used as the base for creating nicer alternatives for other methods using the same pattern (with loads of handwaving for subclassing).

There might be problem here though: NSError** output arguments used by numerous classes, e.g. -[NSAttributedString initWithURL:options:documentAttributes:error:. Python's new generally returns a value of the type, not a tuple with multiple values. Technically code like this would work, but feels "weird":

value, error = NSAttributedString(url=..., options=..., documentAttributes=..., error=None)

I'm currently inclined to accept this weirdness.

A different possible problem: There's a number of init selectors with unnamed selector parts, e.g. -[DOMObject initEvent:::], those can be handled using positional-only arguments as all of those I've found have exactly 1 named selector fragment at the start.

Current plan is to work on this over the summer with inclusion in PyObjC 10, but this depends a lot on how much free time I'll have over the summer (and how much work there is in adapting to changes in macOS 15).

[ronaldoussoren]

A slightly more serious problem: Longer term I'd prefer to provide wrappers for all methods with a completion handler as async methods that can be awaited for.

For example: https://developer.apple.com/documentation/vision/vncoremlrequest/2890152-initwithmodel?language=objc

Not sure yet how to nicely convert this. Likely by having __new__ return an awaitable that returns self once the completionHandler is called.

An additional problem here: both initWithModel: and initWithModel:completionHandler: exists, making it impossible to leaf off the completionHandler bit and convert that into an awaitable result without picking either option.

First stab at this issue should ignore the completionHandler/awaitable issue and just use completionHandler arguments.

[ronaldoussoren]

I have some code to calculate signatures for __new__, but not yet in a form that can be shared. Also the code doesn't handle unavailable init methods (see #159) because the current metadata tooling doesn't collect that information).

Next step is to generate two sets of output:

1. Submodules using objc.addConvenienceForClass to register an __new__ implementation for all classes (one file per framework binding)
2. One or more ReST files with generated documentation (something similar to what I wrote in [https://github.com/add __new__ NS classes #275#issuecomment-1537359012](a previous comment), but for all generated __new__ methods.

(1) allows for playing with the implementation, while (2) is easier for reviewing the interface.

Once I have a first stab at an implementation for this I'll start a branch.

UPDATE: My in progress script reports on about 3200 'init*' methods. Some of which are duplicates, but this does mean reviewing the generated interfaces won't be trivial.

Also: a number of classes, like NSArray already have a __new__, need to make sure that the proposed generic version is compatibel with the existing interface.

[ronaldoussoren]

Small steps....

The following is partial documentation for NSURL and NSURLAuthenticationChallenge (both without looking at the parent class init methods). Output for (again without inherited init methods) is about 16K lines for all framework bindings, with a similar size of (unoptimised) __new__ implementations.

.. class:: NSURL
   .. method:: __new__(*, absoluteURLWithDataRepresentation, relativeToURL)

      Equivalent to ``NSURL.alloc().initAbsoluteURLWithDataRepresentation_relativeToURL_(absoluteURLWithDataRepresentation, relativeToURL)``

   .. method:: __new__(*, byResolvingBookmarkData, options, relativeToURL, bookmarkDataIsStale, error=None)

      Equivalent to ``NSURL.alloc().initByResolvingBookmarkData_options_relativeToURL_bookmarkDataIsStale_error_(byResolvingBookmarkData, options, relativeToURL, bookmarkDataIsStale, error)``

   .. method:: __new__(*, dataRepresentation, relativeToURL)

      Equivalent to ``NSURL.alloc().initWithDataRepresentation_relativeToURL_(dataRepresentation, relativeToURL)``

   .. method:: __new__(*, fileURLWithFileSystemRepresentation, isDirectory, relativeToURL)

      Equivalent to ``NSURL.alloc().initFileURLWithFileSystemRepresentation_isDirectory_relativeToURL_(fileURLWithFileSystemRepresentation, isDirectory, relativeToURL)``

   .. method:: __new__(*, fileURLWithPath)

      Equivalent to ``NSURL.alloc().initFileURLWithPath_(fileURLWithPath)``

   .. method:: __new__(*, fileURLWithPath, isDirectory)

      Equivalent to ``NSURL.alloc().initFileURLWithPath_isDirectory_(fileURLWithPath, isDirectory)``

   .. method:: __new__(*, fileURLWithPath, isDirectory, relativeToURL)

      Equivalent to ``NSURL.alloc().initFileURLWithPath_isDirectory_relativeToURL_(fileURLWithPath, isDirectory, relativeToURL)``

   .. method:: __new__(*, fileURLWithPath, relativeToURL)

      Equivalent to ``NSURL.alloc().initFileURLWithPath_relativeToURL_(fileURLWithPath, relativeToURL)``

   .. method:: __new__(*, scheme, host, path)

      Equivalent to ``NSURL.alloc().initWithScheme_host_path_(scheme, host, path)``

   .. method:: __new__(*, string)

      Equivalent to ``NSURL.alloc().initWithString_(string)``

   .. method:: __new__(*, string, relativeToURL)

      Equivalent to ``NSURL.alloc().initWithString_relativeToURL_(string, relativeToURL)``


.. class:: NSURLAuthenticationChallenge
   .. method:: __new__(*, authenticationChallenge, sender)

      Equivalent to ``NSURLAuthenticationChallenge.alloc().initWithAuthenticationChallenge_sender_(authenticationChallenge, sender)``

   .. method:: __new__(*, protectionSpace, proposedCredential, previousFailureCount, failureResponse, error, sender)

      Equivalent to ``NSURLAuthenticationChallenge.alloc().initWithProtectionSpace_proposedCredential_previousFailureCount_failureResponse_error_sender_(protectionSpace, proposedCredential, previousFailureCount, failureResponse, error, sender)``

The logic is not yet ideal, see the inconsistency in naming for the first variant for VNVector.__new__:

.. class:: VNVector
   .. method:: __new__(*, XComponent, yComponent)

      Equivalent to ``VNVector.alloc().initWithXComponent_yComponent_(XComponent, yComponent)``

   .. method:: __new__(*, r, theta)

      Equivalent to ``VNVector.alloc().initWithR_theta_(r, theta)``

   .. method:: __new__(*, vectorHead, tail)

      Equivalent to ``VNVector.alloc().initWithVectorHead_tail_(vectorHead, tail)``

I've also not yet looked into consistency with the manual __new__ helpers for a number of classes (as mentioned in my previous comment)

[ronaldoussoren]

I've started branch gh-275 to implement this, with a first commit in 334a7a6.

The first commit adds the basic machinery without tests and with just enough support data to call classes as an alternative to calling SomeClass.alloc().init().

[ronaldoussoren]

For python subclasses the keyword arguments are automatically calculated, the following now works in the branch:

class MyObject(NSObject):
    def initWithX_y_(self, x, y):
        self = super().init() 
        self.x = x
        self.y = y
        return self
        
o = MyObject(x=4, y=5)
print(o.x, o.y)

This still needs some work to sync up with the final algorithm to calculate keyword arguments (see note about VNVector in an earlier comment).

[ronaldoussoren]

The branch is now basically finished:

• The generic new is complete and tested
• The few classes that already had a __new__ have been updated, with the exception of NSDictionary and NSMutableDictionary where the generic new conflicts with having a python-like interface.
• Feature is documented

The only thing left to do is update the framework bindings with metadata for setting up the accepted keyword arguments for system classes. That will be done after the merge into the master branch.

From the documentation:

• Every instance selector of the Objective-C with a name starting
  with init adds a possible set of keyword arguments using
  the following algorithm:

  1. Strip initWith or init from the start of the selector;

  2. Lowercase the first character of the result

  3. All segments are now keyword only arguments, in that order.

  For example given, -[SomeClass initWithX:y:z] the
  following invocation is valid: SomeClass(x=1, y=2, z=3).
  Using the keywords in a different order is not valid.

[ronaldoussoren]

Two steps left to do for this issue:

1. Regenerate framework metadata (which will add the generic new keyword sets)
2. (Optionally): Rework documentation to mention the __new__ signatures

[ronaldoussoren]

I'm closing this issue because the required changes have been done in the master branch and will be in the next release (waiting for some minor issue with completely regenerated metadata).