Adjust pins for torch, numpy, other dependencies

[ebr]

Summary

Dependency resolution by pip is fairly lax, and allows fallback to the main PyPi index in case a matching distribution cannot be found. As we're working on an improved installation experience, this is becoming a blocker for installing Invoke using other tools, such as uv. Also, this work exposed some spots where some transitive dependencies might be overridden in spite of different version requirements by direct dependencies (e.g. requests was pinned to a version that is no longer valid).

This PR makes the minimal adjustments to the dependency pins, just to satisfy all direct dependencies and allow the package to be installed.

QA Instructions

This is best tested using a tool that has follows strict dependency resolution rules, such as Astral's uv: https://github.com/astral-sh/uv. Install it prior to testing, then run the following, adjusting paths as necessary:

# breaking test

mkdir -p /tmp/invokeai
uv venv /tmp/invokeai/.venv --python 3.11
cd /tmp/invokeai
. .venv/bin/activate
uv pip install invokeai==5.3.0 --extra-index-url https://download.pytorch.org/whl/cu124

# the above will produce an error to the effect of not being able to find the requested version of numpy

# reset
deactivate; rm -rf /tmp/invokeai/.venv

# repeat with this PR
uv venv /tmp/invokeai/.venv --python 3.11
cd /tmp/invokeai
. .venv/bin/activate
uv pip install git+https://github.com/invoke-ai/InvokeAI.git@ebr/unpin-numpy --extra-index-url https://download.pytorch.org/whl/cu124

# this should work without an issue

Merge Plan

Merge when sufficiently tested on the following platforms / pytorch indices:

windows

• --extra-index-url https://download.pytorch.org/whl/cu124
• --extra-index-url https://download.pytorch.org/whl/cpu

linux

• --extra-index-url https://download.pytorch.org/whl/cu124
• --extra-index-url https://download.pytorch.org/whl/cpu

macOS

• --extra-index-url https://download.pytorch.org/whl/cpu
• no extra index

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)

[psychedelicious]

Testing on MPS:

• The first set of commands fails as-is. I get the expected numpy error.
• The first set of commands succeeds if I remove the extra index URL.
• The second set of commands fails as-is, with a different error:

  uv pip install git+https://github.com/invoke-ai/InvokeAI.git@ebr/unpin-numpy --extra-index-url https://download.pytorch.org/whl/cu124
  Using CPython 3.11.8 interpreter at: /opt/homebrew/opt/python@3.11/bin/python3.11
  Creating virtual environment at: /tmp/invokeai/.venv
  Activate with: source /tmp/invokeai/.venv/bin/activate
  Updated https://github.com/invoke-ai/InvokeAI.git (b39e72dad)
  × No solution found when resolving dependencies:
  ╰─▶ Because only the following versions of torch are available:
          torch<=1.12.0
          torch==1.12.1
          torch==1.13.0
          torch==1.13.1
          torch==2.0.0
          torch==2.0.1
          torch==2.4.0
          torch==2.4.0+cu124
          torch==2.4.1
          torch==2.4.1+cu124
          torch>2.5.0
      and torch>=1.12.0,<=2.0.1 has no wheels with a matching Python ABI tag, we can conclude that torch>=1.12.0,<1.12.1 cannot be used.
      And because torch>=2.4.0,<=2.4.1+cu124 has no wheels with a matching Python implementation tag, we can conclude that torch>=1.12.0,<2.4.0+cu124 cannot be used.
      And because pytorch-lightning==2.1.3 depends on torch>=1.12.0 and diffusers[torch]==0.31.0 depends on torch>=1.4,<2.5.0, we can conclude that pytorch-lightning==2.1.3 and diffusers[torch]==0.31.0 are incompatible.
      And because invokeai==5.3.1rc1 depends on diffusers[torch]==0.31.0 and pytorch-lightning==2.1.3, we can conclude that invokeai==5.3.1rc1 cannot be used.
      And because only invokeai==5.3.1rc1 is available and you require invokeai, we can conclude that your requirements are unsatisfiable.
  
      hint: An index URL (https://download.pytorch.org/whl/cu124) could not be queried due to a lack of valid authentication credentials (403 Forbidden).
  
      hint: `torch` was found on https://download.pytorch.org/whl/cu124, but not at the requested version (all of:
          torch>1.12.0,<1.12.1
          torch>1.12.1,<1.13.0
          torch>1.13.0,<1.13.1
          torch>1.13.1,<2.0.0
          torch>2.0.0,<2.0.1
          torch>2.0.1,<2.4.0
          torch>2.4.0,<2.4.0+cu124
          torch>2.4.0+cu124,<2.4.1
          torch>2.4.1,<2.4.1+cu124
          torch>2.4.1+cu124,<2.5.0
      ). A compatible version may be available on a subsequent index (e.g., https://pypi.org/simple). By default, uv will only consider versions that are published on the first index that contains a given package, to avoid dependency confusion attacks. If all indexes are equally trusted, use `--index-strategy unsafe-best-match` to consider all versions from all indexes, regardless of the order in which they were
      defined.
  

• The second set of commands succeeds if I remove the extra index URL.
• I installed this PR's branch using pip without issue. I built the UI and am able to generate. I only tested SD1.5.

[ebr]

On MPS, it would be good to test with --extra-index-url https://download.pytorch.org/whl/cpu (cpu vs cu124). I updated the testing instructions to clarify. But it's also possible that we explicitly should not use an extra index with MPS. If it succeeded for you without the index, I think we can just stick with that for macOS.

Thanks for confirming that pip works for this branch!

[maryhipp]

@ebr worked great on windows with extra index cu124