Skip to content

Commit

Permalink
TalkBack support for ScrollView accessibility announcements (list and…
Browse files Browse the repository at this point in the history 
… grid) (facebook#33180)

Summary:
This issue fixes [30977][17] . The Pull Request was previously published by [intergalacticspacehighway][13] with [31666][19].
The solution consists of:
1. Adding Javascript logic in the [FlatList][14], SectionList, VirtualizedList components to provide accessibility information (row and column position) for each cell in the method [renderItem][20] as a fourth parameter [accessibilityCollectionItem][21]. The information is saved on the native side in the AccessibilityNodeInfo and announced by TalkBack when changing row, column, or page ([video example][12]). The prop accessibilityCollectionItem is available in the View component which wraps each FlatList cell.
2. Adding Java logic in [ReactScrollView.java][16] and HorizontalScrollView to announce pages with TalkBack when scrolling up/down. The missing AOSP logic in [ScrollView.java][10] (see also the [GridView][11] example) is responsible for announcing Page Scrolling with TalkBack.

Relevant Links:
x [Additional notes on this PR][18]
x [discussion on the additional container View around each FlatList cell][22]
x [commit adding prop getCellsInItemCount to VirtualizedList][23]

[Android] [Added] - Accessibility announcement for list and grid in FlatList

Pull Request resolved: facebook#33180

Test Plan:
[1]. TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer ([link][1])
[2]. TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer ([link][2])
[3]. `FlatList numColumns={undefined}` Should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack. ([link][3])
[4]. TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app ([link][4])

[1]: fabOnReact/react-native-notes#6 (comment)
[2]: fabOnReact/react-native-notes#6 (comment)
[3]: fabOnReact/react-native-notes#6 (comment)
[4]: fabOnReact/react-native-notes#6 (comment)
[10]:https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L1027-L1029 "GridView.java method responsible for calling setFromIndex and setToIndex"
[11]:fabOnReact/react-native-notes#6 (comment) "test case on Android GridView"
[12]:fabOnReact/react-native-notes#6 (comment) "TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer"
[13]:https://github.com/intergalacticspacehighway "github intergalacticspacehighway"
[14]:https://github.com/fabriziobertoglio1987/react-native/blob/80acf523a4410adac8005d5c9472fb87f78e12ee/Libraries/Lists/FlatList.js#L617-L636 "FlatList accessibilityCollectionItem"
[16]:https://github.com/fabriziobertoglio1987/react-native/blob/5706bd7d3ee35dca48f85322a2bdcaec0bce2c85/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L183-L184 "logic added to ReactScrollView.java"
[17]: facebook#30977
[18]: fabOnReact/react-native-notes#6
[19]: facebook#31666
[20]: https://reactnative.dev/docs/next/flatlist#required-renderitem "FlatList renderItem documentation"
[21]: fabOnReact@7514735 "commit that introduces fourth param accessibilityCollectionItem in callback renderItem"
[22]: facebook#33180 (comment) "discussion on the additional container View around each FlatList cell"
[23]: fabOnReact@d50fd1a "commit adding prop getCellsInItemCount to VirtualizedList"

Reviewed By: kacieb

Differential Revision: D34518929

Pulled By: blavalla

fbshipit-source-id: 410a05263a56162bf505a4cad957b24005ed65ed
  • Loading branch information
@fabOnReact @Saadnajmi
fabOnReact authored and Saadnajmi committed Jan 14, 2023
1 parent 504a0b0 commit 76d7d14
Show file tree
Hide file tree
Showing 24 changed files with 1,051 additions and 52 deletions.
1 change: 1 addition & 0 deletions Libraries/Components/View/ViewAccessibility.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export type AccessibilityRole =
| 'tablist'
| 'timer'
| 'list'
| 'grid'
| 'toolbar'
| 'menubutton'; // [macOS]

Expand Down
17 changes: 17 additions & 0 deletions Libraries/Components/View/ViewPropTypes.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -577,6 +577,23 @@ export type ViewProps = $ReadOnly<{|
*/
accessibilityActions?: ?$ReadOnlyArray<AccessibilityActionInfo>,

/**
*
* Node Information of a FlatList, VirtualizedList or SectionList collection item.
* A collection item starts at a given row and column in the collection, and spans one or more rows and columns.
*
* @platform android
*
*/
accessibilityCollectionItem?: ?{
rowIndex: number,
rowSpan: number,
columnIndex: number,
columnSpan: number,
heading: boolean,
itemIndex: number,
},

/**
* Specifies the nativeID of the associated label text. When the assistive technology focuses on the component with this props, the text is read aloud.
*
Expand Down
10 changes: 9 additions & 1 deletion Libraries/Lists/FlatList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -660,11 +660,18 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
return (
<View style={StyleSheet.compose(styles.row, columnWrapperStyle)}>
{item.map((it, kk) => {
const itemIndex = index * cols + kk;
const accessibilityCollectionItem = {
...info.accessibilityCollectionItem,
columnIndex: itemIndex % cols,
itemIndex: itemIndex,
};
const element = renderer({
item: it,
index: index * cols + kk,
index: itemIndex,
isSelected: info.isSelected, // [macOS]
separators: info.separators,
accessibilityCollectionItem,
});
return element != null ? (
<React.Fragment key={kk}>{element}</React.Fragment>
Expand Down Expand Up @@ -695,6 +702,7 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
return (
<VirtualizedList
{...restProps}
numColumns={numColumns}
getItem={this._getItem}
getItemCount={this._getItemCount}
keyExtractor={this._keyExtractor}
Expand Down
67 changes: 65 additions & 2 deletions Libraries/Lists/VirtualizedList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ const FillRateHelper = require('./FillRateHelper');
const Platform = require('../Utilities/Platform'); // [macOS]
const ReactNative = require('../Renderer/shims/ReactNative');
const RefreshControl = require('../Components/RefreshControl/RefreshControl');
const Platform = require('../Utilities/Platform');
const ScrollView = require('../Components/ScrollView/ScrollView');
const StyleSheet = require('../StyleSheet/StyleSheet');
const View = require('../Components/View/View');
Expand Down Expand Up @@ -54,11 +55,21 @@ export type Separators = {
...
};

export type AccessibilityCollectionItem = {
itemIndex: number,
rowIndex: number,
rowSpan: number,
columnIndex: number,
columnSpan: number,
heading: boolean,
};

export type RenderItemProps<ItemT> = {
item: ItemT,
index: number,
isSelected: ?boolean, // [macOS]
separators: Separators,
accessibilityCollectionItem: AccessibilityCollectionItem,
...
};

Expand Down Expand Up @@ -93,9 +104,19 @@ type RequiredProps = {|
*/
getItem: (data: any, index: number) => ?Item,
/**
* Determines how many items are in the data blob.
* Determines how many items (rows) are in the data blob.
*/
getItemCount: (data: any) => number,
/**
* Determines how many cells are in the data blob
* see https://bit.ly/35RKX7H
*/
getCellsInItemCount?: (data: any) => number,
/**
* The number of columns used in FlatList.
* The default of 1 is used in other components to calculate the accessibilityCollection prop.
*/
numColumns?: ?number,
|};
type OptionalProps = {|
renderItem?: ?RenderItemType<Item>,
Expand Down Expand Up @@ -360,6 +381,10 @@ type Props = {|
...OptionalProps,
|};

function numColumnsOrDefault(numColumns: ?number) {
return numColumns ?? 1;
}

let _usedIndexForKey = false;
let _keylessItemComponentName: string = '';

Expand Down Expand Up @@ -1338,6 +1363,17 @@ class VirtualizedList extends React.PureComponent<Props, State> {
);
}

_getCellsInItemCount = props => {
const {getCellsInItemCount, data} = props;
if (getCellsInItemCount) {
return getCellsInItemCount(data);
}
if (Array.isArray(data)) {
return data.length;
}
return 0;
};

_defaultRenderScrollComponent = props => {
// [macOS
const preferredScrollerStyleDidChangeHandler =
Expand All @@ -1362,7 +1398,21 @@ class VirtualizedList extends React.PureComponent<Props, State> {
onKeyDown: this._handleKeyDown,
};
// macOS]
const {getItemCount, data} = props;
const onRefresh = props.onRefresh;
const numColumns = numColumnsOrDefault(props.numColumns);
const accessibilityRole = Platform.select({
android: numColumns > 1 ? 'grid' : 'list',
});
const rowCount = getItemCount(data);
const accessibilityCollection = {
// over-ride _getCellsInItemCount to handle Objects or other data formats
// see https://bit.ly/35RKX7H
itemCount: this._getCellsInItemCount(props),
rowCount,
columnCount: numColumns,
hierarchical: false,
};
if (this._isNestedWithSameOrientation()) {
// $FlowFixMe[prop-missing] - Typing ReactNativeComponent revealed errors
return <View {...props} />;
Expand All @@ -1383,6 +1433,8 @@ class VirtualizedList extends React.PureComponent<Props, State> {
preferredScrollerStyleDidChangeHandler
} // macOS]
{...props}
accessibilityRole={accessibilityRole}
accessibilityCollection={accessibilityCollection}
refreshControl={
props.refreshControl == null ? (
<RefreshControl
Expand All @@ -1398,7 +1450,7 @@ class VirtualizedList extends React.PureComponent<Props, State> {
);
} else {
return (
// $FlowFixMe Invalid prop usage
// $FlowFixMe[prop-missing] Invalid prop usage
<ScrollView
// [macOS
{...(props.enableSelectionOnKeyPress && keyboardNavigationProps)}
Expand All @@ -1407,6 +1459,8 @@ class VirtualizedList extends React.PureComponent<Props, State> {
preferredScrollerStyleDidChangeHandler
} // macOS]
{...props}
accessibilityRole={accessibilityRole}
accessibilityCollection={accessibilityCollection}
/>
);
}
Expand Down Expand Up @@ -2231,11 +2285,20 @@ class CellRenderer extends React.Component<
}

if (renderItem) {
const accessibilityCollectionItem = {
itemIndex: index,
rowIndex: index,
rowSpan: 1,
columnIndex: 0,
columnSpan: 1,
heading: false,
};
return renderItem({
item,
index,
isSelected, // [macOS]
separators: this._separators,
accessibilityCollectionItem,
});
}

Expand Down
16 changes: 15 additions & 1 deletion Libraries/Lists/VirtualizedSectionList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import invariant from 'invariant';
import type {ViewToken} from './ViewabilityHelper';
import type {SelectedRowIndexPathType} from './VirtualizedList'; // [macOS]
import type {KeyEvent} from '../Types/CoreEventTypes'; // [macOS]
import type {AccessibilityCollectionItem} from './VirtualizedList';
import {keyExtractor as defaultKeyExtractor} from './VirtualizeUtils';
import {View, VirtualizedList} from 'react-native';
import * as React from 'react';
Expand Down Expand Up @@ -508,7 +509,16 @@ class VirtualizedSectionList<

_renderItem =
(listItemCount: number) =>
({item, index}: {item: Item, index: number, ...}) => {
({
item,
index,
accessibilityCollectionItem,
}: {
item: Item,
index: number,
accessibilityCollectionItem: AccessibilityCollectionItem,
...
}) => {
const info = this._subExtractor(index);
if (!info) {
return null;
Expand Down Expand Up @@ -537,6 +547,7 @@ class VirtualizedSectionList<
LeadingSeparatorComponent={
infoIndex === 0 ? this.props.SectionSeparatorComponent : undefined
}
accessibilityCollectionItem={accessibilityCollectionItem}
cellKey={info.key}
index={infoIndex}
isSelected={this._isItemSelected(item)} // [macOS]
Expand Down Expand Up @@ -651,6 +662,7 @@ type ItemWithSeparatorProps = $ReadOnly<{|
updatePropsFor: (prevCellKey: string, value: Object) => void,
renderItem: Function,
inverted: boolean,
accessibilityCollectionItem: AccessibilityCollectionItem,
|}>;

function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
Expand All @@ -669,6 +681,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
index,
section,
inverted,
accessibilityCollectionItem,
} = props;

const [leadingSeparatorHiglighted, setLeadingSeparatorHighlighted] =
Expand Down Expand Up @@ -743,6 +756,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
isSelected, // [macOS]
section,
separators,
accessibilityCollectionItem,
});
const leadingSeparator = LeadingSeparatorComponent != null && (
<LeadingSeparatorComponent
Expand Down
59 changes: 59 additions & 0 deletions Libraries/Lists/__tests__/__snapshots__/FlatList-test.js.snap
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ exports[`FlatList renders all the bells and whistles 1`] = `
ListEmptyComponent={[Function]}
ListFooterComponent={[Function]}
ListHeaderComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 5,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand All @@ -29,6 +37,7 @@ exports[`FlatList renders all the bells and whistles 1`] = `
getItemCount={[Function]}
getItemLayout={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -121,6 +130,14 @@ exports[`FlatList renders all the bells and whistles 1`] = `

exports[`FlatList renders empty list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 0,
"rowCount": 0,
}
}
data={Array []}
getItem={[Function]}
getItemCount={[Function]}
Expand All @@ -144,6 +161,14 @@ exports[`FlatList renders empty list 1`] = `

exports[`FlatList renders null list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 0,
"rowCount": 0,
}
}
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
Expand All @@ -166,6 +191,14 @@ exports[`FlatList renders null list 1`] = `

exports[`FlatList renders simple list (multiple columns) 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 3,
"rowCount": 2,
}
}
data={
Array [
Object {
Expand All @@ -182,6 +215,7 @@ exports[`FlatList renders simple list (multiple columns) 1`] = `
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -237,6 +271,14 @@ exports[`FlatList renders simple list (multiple columns) 1`] = `

exports[`FlatList renders simple list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 3,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand Down Expand Up @@ -298,6 +340,14 @@ exports[`FlatList renders simple list 1`] = `
exports[`FlatList renders simple list using ListItemComponent (multiple columns) 1`] = `
<RCTScrollView
ListItemComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 3,
"rowCount": 2,
}
}
data={
Array [
Object {
Expand All @@ -314,6 +364,7 @@ exports[`FlatList renders simple list using ListItemComponent (multiple columns)
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -369,6 +420,14 @@ exports[`FlatList renders simple list using ListItemComponent (multiple columns)
exports[`FlatList renders simple list using ListItemComponent 1`] = `
<RCTScrollView
ListItemComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 3,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand Down
Loading

0 comments on commit 76d7d14

Please sign in to comment.