Skip to content

[add] merge two onnx and related #131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 0 commits into from

Conversation

xiaominghan121
Copy link

提供合并两个ONNX模型的功能,同时支持多个节点复制和删除功能。
可以满足https://github.com/ZhangGe6/onnx-modifier/issues/63需求。

@ZhangGe6
Copy link
Owner

Thanks for your effort! I am reviewing the codes. This may take some time as this PR contains many modifications.

Copy link
Owner

@ZhangGe6 ZhangGe6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution. I have reviewed the codes and left some suggestions/confusing points. Please check. In addition, this patch is named with "merge two onnx and related" but ships with features including including but not limited to: 1) merge models; 2) copy nodes; 3) other minor modifications. It is suggested to break them into separate patches. Thanks for your effort again!

static/onnx.js Outdated
if (!map.has(item.name)) {
map.set(item.name, []);
}
map.get(item.name).push(item);
}

// reorder the map
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why reorder here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When adding a node, convolutional nodes and several other types are more commonly used. It is more convenient to have them displayed at the beginning of the node list.

onnx_modifier.py Outdated
logging.info("load done!")
return cls(name, model_proto)

@classmethod
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To merge two models, it is expected to connect some of the outputs from the first model with inputs from the second model. However, this logic is not implemented in the merge() function.

reference: https://github.com/onnx/onnx/blob/main/docs/PythonAPIOverview.md#onnx-compose

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I have considered this issue, but the problem encountered by the person who proposed this issue is just one of many requirements involving merging two models. We may encounter different needs such as placing the main structures of two models side by side with shared inputs and outputs; or we might need to replace a part of an existing model with a portion of another model, and so on. Therefore, considering comprehensively, I merely put the two models together. As for how to proceed next, adjustments will still need to be made based on specific scenarios and cannot be achieved all at once. This is a trade-off I made based on minimizing code changes.

In fact, a more elegant solution would be to support cross-file module copy and paste functionality, but that might require a significant amount of time to implement, so it has not been realized.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, merging models can involve many cases. I think it can be a user-driven feature, and we support the cases that users call for at a higher priority, such as connecting models with one's tail and another's head.

README_zh-CN.md Outdated
@@ -182,6 +182,19 @@

`onnx-modifer`正在活跃地更新中:hammer_and_wrench:。 欢迎使用,提issue,如果有帮助的话,感谢给个:star:~

## 合并两个onnx模型
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move these sections ahead of "onnx-modifer正在活跃地更新中...".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, my mistake.

README_zh-CN.md Outdated
> 1. 只能复制已知节点,未知节点需要先修改`metadata.json`,即先变为已知节点。
> 2. 调试模式下按下Alt,进入断点并松开Alt,退出调试后,Alt会被认为一直是按下状态。需要重新按一次解除。

## 复制或删除多个节点
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting nodes is already supported. What is the motivation of "deleting nodes" here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Umm, my description here might not be very precise. What I have implemented is the ability to select two nodes as the start and end points, thereby selecting all consecutive nodes between them, and supporting deletion and duplication of these selected nodes. This feature facilitates adjustments to the model structure. Previously, the deletion function was limited to removing a single node or all subsequent nodes, so there is a little difference in functionality.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. It is suggested to split this feature into a separate patch. To be frank, I am not sure whether it is a commonly used feature.


getPathNodeNames(start, end)
{
var twoDimensionalArray = new Set();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A more meaningful variable name is expected.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right. This is the most challenging task, and I need to think it over carefully.

@xiaominghan121
Copy link
Author

Thank you for your reply. I have responded to your code review and explained some of my design thoughts. I am not sure if my approach is appropriate. I hope to receive your feedback so that I can make the necessary adjustments or modifications.

@xiaominghan121
Copy link
Author

按照 #131 (review) 建议将各个功能分离成不同提交,新的PR已提交 #135

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants