Processing 4

Some of these items have been started, or are covered in more detail on the Project List page.

Changes under the hood

• Move to Java 17 (done)

  • Some of this work is in progress
  • If we move to Java 11, we will drop support for Java 8. We have no resources to do backwards compatibility, and need to make things less complicated.
  • Done: We're now using Java 11, and in September 2021, we'll use Java 17.
  • As of 4.0 beta 3, we're using Java 17.

• Drop most 32-bit support (done)

  • People who need 32-bit support can use Processing 3.
  • This is a bummer for the ARM side of things, where most devices are still 32-bit. Because we'll have to support Apple Silicon as a variant, the code for “variants” of a platform will still be maintained, so we'd like to keep supporting Raspberry Pi devices which are still overwhelmingly 32-bit.
  • Done: The build process is currently 64-bit only, and there are no plans to change that.

• Change base platforms that are supported/tested (in progress)

  1. macOS 10.15 (Catalina), 11 (Big Sur), 12 (Monterey) on Intel hardware – Apple is upgrading once a year, we are forced to chase them. Major OS releases are supported just 3 years, we shouldn't (can't, for lack of time) be extending beyond that.
  2. macOS 12 (Monterey) on Apple Silicon – we don't have support for this yet, but would like to before the final version of 4.0. This will be a separate download. We're now shipping an Apple Silicon version as of beta 8! A few more tidbits remain but we're mostly there.
  3. Windows 10 (64-bit) – support for Windows 7 ended in 2018. We are not currently (as of 20 January 2022) testing on Windows 11 and therefore cannot recommend it.
  4. Linux (Ubuntu 22.04, 64-bit) – will change to 22.04 as soon as it's available in April. If you're using a different Linux, please help us make improvements so that it runs on your platform of choice. We just can't do everything.
  5. Linux on Raspberry Pi (32- and 64-bit) – We don't currently have a maintainer, but we're doing releases for 32- and 64-bit. Please help!

• Clean up per-platform handling inside Libraries (done in beta 4)

  • We started Processing in 2001 with windows, linux, macos9, and macosx. We later dropped macos9. Then we split things into 32- and 64-bit with windows32, windows64, linux32, and linux64. macosx was unchanged because it included both PPC and Intel versions of the software, and then later 32-bit and 64-bit Intel versions of the software. Then we hacked in support for Raspberry Pi with linux-armv6hf and linux-arm64.
  • Starting in beta 4, these will be cleaned up to use the platform name, plus the os.arch property for that platform.
    • macosx becomes macos-x86_64 and macos-aarch64. The x also goes away because it's now macOS not Mac OS X. The x86_64 suffix is unfortunate, but that's what Apple chose to call the 64-bit version their Intel hardware.
    • windows64 becomes windows-amd64, and is the only supported Windows platform.
    • linux64 becomes linux-amd64.
    • linux-armv6hf becomes linux-arm (32-bit RPi Linux) and linux-arm64 is linux-aarch64.
  • More at the Supported Platforms page.

Renderers

• Switch OpenGL from JOGL to LWJGL Update JOGL to 2.4 (resolved)

  • The JOGL project hasn't been receiving updates lately, but LWJGL is quite active.
  • It may also make sense to move OpenGL back out of core as a result.
  • We're currently using the most recent RC or “release candidate” build of JOGL. It already fixes many issues that folks are having with 3.5.4.
  • Status: it sounds like JOGL 2.4 is likely to happen soon (before the end of 2020?), so we'd like to stick with JOGL for 4.0, to maintain compatibility for existing sketches.

• Include an LWJGL renderer (won't be happening for 4.x)

  • This work has been started in a branch, but needs to be completed. It'll require some PDE modifications to make it happen, due to threading changes (different options to be passed to the JVM on launch).
  • This has been in progress, and might be useful as a future default (or recommendation), but let's ship a version

• Move JavaFX out of core and into a Library (done)

  • Oracle has removed it from the JDK, so it's now a heavyweight addition.
  • This would simply mean using Import Library > JavaFX in sketches.
  • Our intent was to eventually replace the “default” (Java2D) renderer with JavaFX. However, given the quirks of JavaFX and lack of time, we've never gotten there. With Oracle kicking JavaFX out of the main distribution, its future is also muddy, so we probably won't be able to rely upon it this way.
  • Unfortunately this breaks the two Tools that use JavaFX, but it's not worth doubling the download size of Processing itself.
  • As of 4.0 beta 4, JavaFX will be available from the Contributions Manager. #348

Rebuilding the PDE

• Rebuild the preprocessor (done)

  • It's a little fragile and newer Java language features are not supported.
  • Status: done! Sam Pottinger has completely rebuilt the preprocessor for us to support new language features.

• Use a new Editor text area component (nope, see below)

  • This would give us a better Undo implementation, and fewer quirks in the editor.
  • Attempted this with RSyntaxTextArea, but the changes are invasive (breaks all Modes, requires rewriting custom UI)
  • The amount of time necessary for the changes is probably better spent on a Language Server implementation (see below)
  • Final decision: after spending several more hours exploring a move to RSyntaxArea, it's just not gonna happen. The gory details can be found in the README here.

• Add a Language Server implementation for the preprocessor, compiler, etc.

  • This would allow Processing to work with other editors, reducing the stress on the PDE needing to support every last detail that advanced users seek.
  • This requires a full refactor of Java Mode, to separate the UI and code handling functions, and exposing the code handling via a series of Language Server endpoints.
  • More discussion can be seen here.

• Remove requirement for the strict sketch folder layout (done, details below)

  • Make the sketch.properties file ever-present, and don't require the main .pde file to share the sketch name. More likely, it'd be better to use a file with a different name, so that double-clicking it would launch the PDE.
  • This comes from a shift in priorities since we first started. When we began, we didn't even have multiple tabs, and it was common to have single .pde files for entire sketches. We didn't want to make things needlessly complex by adding another file that just pointed to the first file. Nowadays, many/most sketches have multiple tabs, and folks don't mind project files all that much.
  • This is a high priority because having the main .pde named the same thing as its enclosing folder makes version control difficult.
  • Would really like to do this, but it's a potentially massive change. Hard to say how much this might break.
  • Starting in beta 6, it's possible to rename the main tab. We're keeping the default behavior the same as before, so that it doesn't break expectations and documentation, but for people who want this, it's now available.

Things I'd like to do

• Fixed aspect ratio option/“designed” width/height (done in beta 6)

  • Something like aspect(1920, 1080), which would maintain that aspect ratio, no matter how the window might be resized. Mouse coordinates will be scaled to the original code, while aspect ratio is maintained even as the sketch window size is changed. I use this on a regular basis in my own work, but don't have a good idea for how to name it properly, otherwise we'd include it now.
  • It's called windowRatio() in beta 6. Check out the revisions page for more details on how to use it.

• Move more of the utility functions in PApplet into a PUtil class that doesn't require a PApplet instance. I often wind up building quasi versions of this in professional work, which undermines the point of reusable code.

• Move the essentials into a PSketch class that can be the base. Make PApplet a compatibility layer that subclasses PSketch and automatically imports PUtil and anything like it. This would allow other kinds of projects to build against something more streamlined than PApplet without breaking compatibility for others.

• Reworking parts of the processing.data classes for concurrency (one of the primary reasons to use Java for data handling work).

• Implement a single-file archive format for sketches, i.e. .psk (done)

  • This would be a zipped sketch folder, and would allow users to double click a single file, or click a link to one from the web, and install/run the sketch immediately.
  • This todo has been around a long time: https://github.com/processing/processing/issues/73
  • Included in 4.0 beta 1. A .pdez file is a sketch, and a .pdex file can be used to store a Library, Mode, Tool, or Example set. Check out an image of the original issue filed (by me) back in 2005.

• Improving the startup and examples experience for the PDE

  • The “I just opened up this app called Processing” experience isn't great.
  • Have an easier way to get people into the examples and start playing with things.
  • A useful welcome page that also covers recent updates.

• Implement a better way to browse examples

  • We have hundreds of examples and they're central to how people learn. They need to be easier to introduce, navigate, and encourage modification.

Things we should do

• Touch Events

  • High-level events for swipes, taps, etc

• Better handling of surface

  • The surface variable was introduced in 3.x, however we had no idea quite how many people were using those features in their sketches. Because it's so common, we need a better API that doesn't require use of the surface object. (And for more testing across renderers to ensure that those methods work, which is the major reason they live in surface as an advanced feature.)

• Redo JSON implementation

  • Working with JSON in Java is a huge headache. We can do better.

• New vector class

  • PVector is optimized in ways that people don’t really use, making it more confusing than necessary

• Better handling of pixel vs. non-pixel surfaces

  • We started at a time when Processing was about high-performance ways to work with pixel data from surfaces. Nowadays, those features make Processing slower than necessary.
  • We have a hodgepodge of workarounds like loadPixels() and hidden API calls in OpenGL for sending video to textures, but these need to be ironed out better.

• Mutable/immutable versions of the “data” classes

  • Table was built to be mostly immutable, but that's not the way a lot of people use it. Splitting into separate versions would improve performance and make behavior clearer.

• Improve or replace PShape

  • PShape has grown very complex and should probably be handled differently.
  • When started, we were looking for a lightweight (a few Kb) way to do SVG. Nowadays, size footprint isn't much of an issue, and its harder to draw the line on what should be in or out.
  • The class was also designed to be compact, but has since been extended for OBJ and other formats, so it's grown a bit… monolithic.
  • It might be better to get improved SVG support by using something like Batik underneath.

More fun, but even more work

• Automatic version control of sketches

  • Some sort of semi-automated way of running Git behind the scenes for sketches, but without requiring people to learn Git. This would harken back to the “History” feature we started with in 2001, but never finally implemented.

• HTML/CSS/JavaScript version of the PDE

  • If we move to a Language Server implementation, it would be nice to get rid of the Swing-based PDE and rewrite it as a web UI (i.e. an Electron app) that speaks the language server protocol. It's been incredibly difficult to find people to contribute to fixing the Editor, and this would open it up to more contributors. Theia seems like a good starting point.

• Sketchbook sync to Box or S3 or Dropbox or…

• Focus on large-scale, full screen work

  • With many smaller projects using p5.js, the desktop version could instead focus on more advanced use.
  • This might also involve redoing the examples to use a larger area, or at least making them more adjustable.
  • Working from 4K and larger displays

• Better than 8-bit color

  • This is a huge shift, but things are moving quickly in this area, and you can make some really, really beautiful things with HDR, 10-bit color, etc.

• Create paid app store versions of Processing to support the Foundation

  • Build a nice version of the software that's available from the Mac App Store or Windows Store. Using these releases would be optional, but provide a subscription-based way for people to support the project.
  • Multiple projects have released paid, closed-source software that use our API and approach (even to the point of copying our documentation), received significant recognition and revenue for it, and also failed to attribute where it all came from. It'd be nice to at least have a venue for paid support of the project outside of asking for donations.
  • This would automate update handling too, which isn't fun at the moment.
  • We could also add additional features (i.e. the sketchbook sync features mentioned earlier) to this version to encourage folks to use it.

• Generally speaking, covering the split between what can't be done in languages like JavaScript or Python (performance or implementation-wise) and what's still difficult to do with C++ and others.

Goodbye Java?

Sun has never been supportive of our project in spite of the large number of users we bring to the platform. And they've never made up their mind about how much they do/do not care about Java on the desktop. In recent years, it's been even more frustrating watching Oracle make a mess of Java in general.

Unfortunately, there's no obvious language to move to: JavaScript gives us the web, but isn't nearly as performant (e.g. working with a lot of data) or mature (e.g. working with threads, etc) as Java in many places. Python is terrific, but there aren't a lot of readily available pieces in place for doing interactive graphics (i.e. a well-supported, complete OpenGL implementation). When we started in 2001, Java was the best of both worlds (well sorta) because we had decent performance (a lot better than JS or ActionScript) and could easily distribute over the web (with applets).

A few approaches, sometimes with pros and cons:

• A full port of Processing to JavaScript/Node/etc

  • p5.js is not a port (nor should it be)
  • …though done the wrong way, this could be a lot of duplicate effort
  • It would be nice to be able to translate existing projects in a more line-by-line way
  • Sharing via the web is hugely important

• Porting the entire library to Python

  • The Jython implementation is great, but it'd be awesome to have available NumPy and all those other things that make Python wonderful.

• Implement things on Apple's Swift

  • Excellent performance, works on iOS, macOS, watchOS tvOS… but Apple's rules are antithetical to many of our core values. App store approval is a gateway to distribution, everyone must use Xcode, etc.
  • I've made some progress on this, and there's also been a Google Summer of Code project that looked at it, but nothing formal or finished yet.

• Rust seems excellent

  • Provides C/C++ level performance without… making everyone learn C++.