Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading data from a worksheet with huge amounts of data #1220

Closed
ecioc opened this issue May 4, 2022 · 4 comments
Closed

Reading data from a worksheet with huge amounts of data #1220

ecioc opened this issue May 4, 2022 · 4 comments
Labels
needs more info This issue can't reproduce, need more info

Comments

@ecioc
Copy link

ecioc commented May 4, 2022

单个Excel就262个sheet,每个sheet差不多在300*300单元格。
`
(pprof) top
Showing nodes accounting for 998.92MB, 99.15% of 1007.43MB total
Dropped 74 nodes (cum <= 5.04MB)
Showing top 10 nodes out of 37
flat flat% sum% cum cum%
503.05MB 49.93% 49.93% 892MB 88.54% github.com/xuri/excelize/v2.prepareSheetXML
359.95MB 35.73% 85.66% 388.95MB 38.61% github.com/xuri/excelize/v2.fillColumns
39.03MB 3.87% 89.54% 39.03MB 3.87% github.com/xuri/excelize/v2.(*File).saveFileList (inline)
21.50MB 2.13% 91.67% 25.50MB 2.53% encoding/xml.(*printer).marshalAttr
18.50MB 1.84% 93.51% 18.50MB 1.84% github.com/xuri/excelize/v2.ColumnNumberToName
18.38MB 1.82% 95.33% 18.38MB 1.82% golang.org/x/net/webdav.(*memFile).Write
16MB 1.59% 96.92% 985.04MB 97.78% xxxxxxxxxxxxx
13.50MB 1.34% 98.26% 29MB 2.88% github.com/xuri/excelize/v2.CoordinatesToCellName
9MB 0.89% 99.15% 9MB 0.89% strconv.formatBits
0 0% 99.15% 25.50MB 2.53% encoding/xml.(*Encoder).Encode

`

@xuri
Copy link
Member

xuri commented May 5, 2022

Thanks for your feedback. Which version of Excelize are you using? Can you show us a complete, standalone example program or reproducible demo? The library has been provided streaming API to generating or reading data from a worksheet with huge amounts of data, please using the stream API if you can, and here are some performance figures for your reference: Performance Figures.

@xuri xuri added the needs more info This issue can't reproduce, need more info label May 5, 2022
@xuri
Copy link
Member

xuri commented Jun 9, 2022

It seems still not have enough info of code and attachments to reproduce this issue, so I'll close this issue, if you have any questions, please let me know to reopen this anytime.

@xuri xuri closed this as completed Jun 9, 2022
@xuri xuri changed the title 使用过程中,部分func占用内存过高,能否优化一下? Reading data from a worksheet with huge amounts of data Jul 13, 2022
@lovepepsi
Copy link

@ecioc 你好,我们也遇上同样的问题,我们用的版本号是v2.1.0,你们呢?

@xuri
Copy link
Member

xuri commented Jul 21, 2022

Hi @lovepepsi, please upgrade to the master branch code and using the rows iterator read data as streaming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs more info This issue can't reproduce, need more info
Projects
None yet
Development

No branches or pull requests

3 participants