[Feature Request]: Upgrade pandas to 2.x

[AnandInguva]

What would you like to happen?

Recently Pandas released a new major version https://pypi.org/project/pandas/#files.

Update the Pandas bounds in setup.py

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[liferoad]

With pandas 2.0, from apache_beam.runners.interactive.interactive_runner import InteractiveRunner stops working with this error: AttributeError: type object 'Series' has no attribute 'append'. Did you mean: '_append'?

[AnandInguva]

Pandas 2.0 removed the append method for both Series and Dataframe. I think we should do the same(remove append from deferred dataframes) but we need to provide alternatives with concat.

pandas-dev/pandas#35407

cc: @robertwb

[tvalentyn]

Let's track in #27221