From 00fb38ebd6acbe1f9680ebcddce94ddea6a06986 Mon Sep 17 00:00:00 2001 From: hunterowens Date: Mon, 6 Oct 2014 10:59:31 -0500 Subject: [PATCH] DOC: warning about copying for df.append/concat. Fixes #7967 updating concat section to reflex comments on PR #8488 DOC: placing the concat notes in a .. note:: block. #8488 DOC: Minor formatting changes for #8488 --- doc/source/merging.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/doc/source/merging.rst b/doc/source/merging.rst index 922fb84c57a56..7d21ba4a3b694 100644 --- a/doc/source/merging.rst +++ b/doc/source/merging.rst @@ -51,7 +51,6 @@ takes a list or dict of homogeneously-typed objects and concatenates them with some configurable handling of "what to do with the other axes": :: - concat(objs, axis=0, join='outer', join_axes=None, ignore_index=False, keys=None, levels=None, names=None, verify_integrity=False) @@ -100,6 +99,18 @@ means that we can now do stuff like select out each chunk by key: It's not a stretch to see how this can be very useful. More detail on this functionality below. +.. note:: + It is worth noting however, that ``concat`` (and therefore ``append``) makes + a full copy of the data, and that constantly reusing this function can + create a signifcant performance hit. If you need to use the operation over + several datasets, use a list comprehension. + +:: + + frames = [ process_your_file(f) for f in files ] + result = pd.concat(frames) + + Set logic on the other axes ~~~~~~~~~~~~~~~~~~~~~~~~~~~