fix: Table Optimization #5319

colnii · 2025-07-25T09:08:11Z

No description provided.

gru-agent · 2025-07-25T09:08:23Z

TestGru Assignment

Summary

Link	CommitId	Status	Reason
Detail	`8546ce7`	🚫 Skipped	No files need to be tested {"packages/global/common/string/markdown.ts":"File path does not match include patterns.","packages/global/common/string/textSplitter.ts":"File path does not match include patterns.","packages/service/thirdProvider/doc2x/index.ts":"File path does not match include patterns."}

History Assignment

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

github-actions · 2025-07-25T09:10:33Z

Preview mcp_server Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_mcp_server_1da256c51d06a25265e16eb82670e4f2eef3154a

github-actions · 2025-07-25T09:10:38Z

Preview sandbox Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_sandbox_d74bd273385ce42a2e963fbd215659eda22dee87

github-actions · 2025-07-25T09:15:13Z

Preview fastgpt Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_d74bd273385ce42a2e963fbd215659eda22dee87

packages/global/common/string/textSplitter.ts

c121914yu · 2025-07-28T16:00:55Z

packages/service/worker/readFile/extension/xlsx.ts

+      row.map((cell) => (typeof cell === 'string' ? cell.replace(/[\n\r]/g, ' ') : cell))
+    )
+  }));
+


数据类型判断重复，肯定是 string。并且不能把换行换成空格，语义不对。
而且应该在原 replace 替换即可，不应该增加计算复杂度

不换成空格的话，可以换成“<br”>和“\\n”吗，还是说有别的方法

packages/global/common/string/textSplitter.ts

c121914yu · 2025-07-29T09:41:02Z

packages/global/common/string/textSplitter.ts

@@ -468,10 +479,10 @@ export const splitText2Chunks = (props: SplitProps): SplitResponse => {

  const splitResult = splitWithCustomSign.map((item) => {
    if (strIsMdTable(item)) {
-      return markdownTableSplit(props);
+      return markdownTableSplit({ ...props, text: item, tableChunkOverflowRatio: 1.2 });


这里为啥×1.2？

c121914yu · 2025-07-29T09:41:12Z

packages/global/common/string/textSplitter.ts

+        const { chunks: tableChunks } = markdownTableSplit({
+          text: currentText,
+          chunkSize,
+          tableChunkOverflowRatio: 1.2


不用加参数。直接把 props 里的 chunksize×1.2 就好了

c121914yu · 2025-07-29T09:41:59Z

packages/service/worker/readFile/extension/xlsx.ts

@@ -31,7 +31,7 @@ export const readXlsxRawText = async ({
 | ${header.map(() => '---').join(' | ')} |
 ${csvArr
  .slice(1)
-  .map((row) => `| ${row.map((item) => item.replace(/\n/g, '\\n')).join(' | ')} |`)
+  .map((row) => `| ${row.map((item) => item.replace(/[\n\r]/g, ' ')).join(' | ')} |`)


这里不能都改成空格，语义有问题，都要改成转义符

改成转义字符的话'\n'和'<br>'都会错位，之前不是聊了说改空格不影响ai嘛，改成转义符的话怎么改呢

* feat: table test: 1 * feat: table test: 2 * feat: table test: 3 * feat: table test: 4 * feat: table test : 5 把maxSize改回chunkSize * feat: table test : 6 都删了，只看maxSize * feat: table test : 7 恢复初始，接下来删除标签功能 * feat: table test : 8 删除标签功能 * feat: table test : 9 删除标签功能成功 * feat: table test : 10 继续调试，修改trainingStates * feat: table test : 11 修改第一步 * feat: table test : 12 修改第二步 * feat: table test : 13 修改了HtmlTable2Md * feat: table test : 14 修改表头分块规则 * feat: table test : 15 前面表格分的太细了 * feat: table test : 16 改着改着表头又不加了 * feat: table test : 17 用CUSTOM_SPLIT_SIGN不行，重新改 * feat: table test : 18 表头仍然还会多加，但现在分块搞的合理了终于 * feat: table test : 19 还是需要搞好表头问题，先保存一下调试情况 * feat: table test : 20 调试结束，看一下replace有没有问题，没问题就pr * feat: table test : 21 先把注释删了 * feat: table test : 21 注释replace都改了，下面切main分支看看情况 * feat: table test : 22 修改旧文件 * feat: table test : 23 修改测试文件 * feat: table test : 24 xlsx表格处理 * feat: table test : 25 刚才没保存先com了 * feat: table test : 26 fix * feat: table test : 27 先com一版调试 * feat: table test : 28 试试放format2csv里 * feat: table test : 29 xlsx解决 * feat: table test : 30 tablesplit解决 * feat: table test : 31 * feat: table test : 32

* perf: system toolset & mcp (#5200) * feat: support system toolset * fix: type * fix: system tool config * chore: mcptool config migrate * refactor: mcp toolset * fix: fe type error * fix: type error * fix: show version * chore: support extract tool's secretInputConfig out of inputs * chore: compatible with old version mcp * chore: adjust * deps: update dependency @fastgpt-skd/plugin * fix: version * fix: some bug (#5316) * chore: compatible with old version mcp * fix: version * fix: compatible bug * fix: mcp object params * fix: type error * chore: update test cases * chore: remove log * fix: toolset node name * optimize app logs sort (#5310) * log keys config modal * multiple select * api * fontsize * code * chatid * fix build * fix * fix component * change name * log keys config * fix * delete unused * fix * perf: log code * perf: send auth code modal enter press * fix log (#5328) * perf: mcp toolset comment * perf: log ui * remove log (#5347) * doc * fix: action * remove log * fix: Table Optimization (#5319) * feat: table test: 1 * feat: table test: 2 * feat: table test: 3 * feat: table test: 4 * feat: table test : 5 把maxSize改回chunkSize * feat: table test : 6 都删了，只看maxSize * feat: table test : 7 恢复初始，接下来删除标签功能 * feat: table test : 8 删除标签功能 * feat: table test : 9 删除标签功能成功 * feat: table test : 10 继续调试，修改trainingStates * feat: table test : 11 修改第一步 * feat: table test : 12 修改第二步 * feat: table test : 13 修改了HtmlTable2Md * feat: table test : 14 修改表头分块规则 * feat: table test : 15 前面表格分的太细了 * feat: table test : 16 改着改着表头又不加了 * feat: table test : 17 用CUSTOM_SPLIT_SIGN不行，重新改 * feat: table test : 18 表头仍然还会多加，但现在分块搞的合理了终于 * feat: table test : 19 还是需要搞好表头问题，先保存一下调试情况 * feat: table test : 20 调试结束，看一下replace有没有问题，没问题就pr * feat: table test : 21 先把注释删了 * feat: table test : 21 注释replace都改了，下面切main分支看看情况 * feat: table test : 22 修改旧文件 * feat: table test : 23 修改测试文件 * feat: table test : 24 xlsx表格处理 * feat: table test : 25 刚才没保存先com了 * feat: table test : 26 fix * feat: table test : 27 先com一版调试 * feat: table test : 28 试试放format2csv里 * feat: table test : 29 xlsx解决 * feat: table test : 30 tablesplit解决 * feat: table test : 31 * feat: table test : 32 * perf: table split * perf: mcp old version compatibility (#5342) * fix: system-tool secret inputs * fix: rewrite runtime node i18n for system tool * perf: mcp old version compatibility * fix: splitPluginId * fix: old mcp toolId * fix: filter secret key * feat: support system toolset activation * chore: remove log * perf: mcp update * perf: rewrite toolset * fix:delete variable id (#5335) * perf: variable update * fix: multiple select ui * perf: model config move to plugin * fix: var conflit * perf: variable checker * Avoid empty number * update doc time * fix: test * fix: mcp object * update count app * update count app --------- Co-authored-by: Finley Ge <[email protected]> Co-authored-by: heheer <[email protected]> Co-authored-by: heheer <[email protected]> Co-authored-by: colnii <[email protected]> Co-authored-by: dreamer6680 <[email protected]>

colnii added 23 commits July 25, 2025 16:58

feat: table test: 1

6bacb75

feat: table test: 2

51366fc

feat: table test: 3

ed686f9

feat: table test: 4

23fd3b4

feat: table test : 5 把maxSize改回chunkSize

c86e516

feat: table test : 6 都删了，只看maxSize

0b26b41

feat: table test : 7 恢复初始，接下来删除标签功能

6ab2ea7

feat: table test : 8 删除标签功能

672f224

feat: table test : 9 删除标签功能成功

5d4dc3c

feat: table test : 10 继续调试，修改trainingStates

1efde8f

feat: table test : 11 修改第一步

bff245b

feat: table test : 12 修改第二步

9b7cb53

feat: table test : 13 修改了HtmlTable2Md

dea3e38

feat: table test : 14 修改表头分块规则

f98983b

feat: table test : 15 前面表格分的太细了

d104c9d

feat: table test : 16 改着改着表头又不加了

887ec90

feat: table test : 17 用CUSTOM_SPLIT_SIGN不行，重新改

3b6df46

feat: table test : 18 表头仍然还会多加，但现在分块搞的合理了终于

86afe3d

feat: table test : 19 还是需要搞好表头问题，先保存一下调试情况

ac931b3

feat: table test : 20 调试结束，看一下replace有没有问题，没问题就pr

16e800e

feat: table test : 21 先把注释删了

beb3c0c

feat: table test : 21 注释replace都改了，下面切main分支看看情况

d64eceb

feat: table test : 22 修改旧文件

8546ce7

pull-request-size bot added the size/M label Jul 25, 2025

colnii changed the title ~~Table test~~ feat: Table Optimization Jul 25, 2025

colnii changed the title ~~feat: Table Optimization~~ fix: Table Optimization Jul 25, 2025

feat: table test : 23 修改测试文件

77c1d30

pull-request-size bot added size/L and removed size/M labels Jul 25, 2025

colnii added 2 commits July 28, 2025 14:37

feat: table test : 24 xlsx表格处理

1911056

feat: table test : 25 刚才没保存先com了

110cff2

c121914yu reviewed Jul 28, 2025

View reviewed changes

packages/global/common/string/textSplitter.ts Outdated Show resolved Hide resolved

feat: table test : 26 fix

3d87caf

c121914yu reviewed Jul 29, 2025

View reviewed changes

colnii added 6 commits July 30, 2025 11:28

feat: table test : 27 先com一版调试

2231913

feat: table test : 28 试试放format2csv里

ad38eb8

feat: table test : 29 xlsx解决

7018126

feat: table test : 30 tablesplit解决

1e2d68e

feat: table test : 31

1da256c

feat: table test : 32

d74bd27

c121914yu approved these changes Jul 30, 2025

View reviewed changes

c121914yu changed the base branch from main to v4.11.1-dev July 30, 2025 07:59

c121914yu merged commit 339195a into labring:v4.11.1-dev Jul 30, 2025
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Table Optimization #5319

fix: Table Optimization #5319

Uh oh!

colnii commented Jul 25, 2025

Uh oh!

gru-agent bot commented Jul 25, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 25, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 25, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

c121914yu Jul 28, 2025

Uh oh!

colnii Jul 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

c121914yu Jul 29, 2025

Uh oh!

This comment was marked as resolved.

Uh oh!

c121914yu Jul 29, 2025

Uh oh!

c121914yu Jul 29, 2025

Uh oh!

colnii Jul 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

fix: Table Optimization #5319

fix: Table Optimization #5319

Uh oh!

Conversation

colnii commented Jul 25, 2025

Uh oh!

gru-agent bot commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TestGru Assignment

Summary

Uh oh!

github-actions bot commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview mcp_server Image:

Uh oh!

github-actions bot commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview sandbox Image:

Uh oh!

github-actions bot commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview fastgpt Image:

Uh oh!

Uh oh!

Uh oh!

c121914yu Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

colnii Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

c121914yu Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

c121914yu Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

c121914yu Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

colnii Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gru-agent bot commented Jul 25, 2025 •

edited

Loading

github-actions bot commented Jul 25, 2025 •

edited

Loading

github-actions bot commented Jul 25, 2025 •

edited

Loading

github-actions bot commented Jul 25, 2025 •

edited

Loading

colnii Jul 29, 2025 •

edited

Loading

colnii Jul 29, 2025 •

edited

Loading