This repository was archived by the owner on Dec 15, 2022. It is now read-only.
Optimize DisplayLayer's memory usage by storing transformations in a Patch #185
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Nathan Sobo <[email protected]>
Signed-off-by: Nathan Sobo <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
If a screen line has a fold, always update the entire line, even if the requested buffer row range starts or ends in the middle of the screen line. If a requested buffer row contains soft wraps, always update all screen lines that belong to that buffer row. Signed-off-by: Antonio Scandurra <[email protected]>
Signed-off-by: Antonio Scandurra <[email protected]>
Signed-off-by: Antonio Scandurra <[email protected]>
We’ll use the native version when we add the necessary features.
Signed-off-by: Antonio Scandurra <[email protected]>
Signed-off-by: Antonio Scandurra <[email protected]>
Signed-off-by: Antonio Scandurra <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Max Brunsfeld <[email protected]>
Signed-off-by: Nathan Sobo <[email protected]>
Signed-off-by: Nathan Sobo <[email protected]>
Signed-off-by: Nathan Sobo <[email protected]>
…for-display-layer
|
Came here via the release notes - does this improve file open time when syntax highlighting is applied as well or does that happen at another layer of the application? |
|
The main improvement is a dramatically lower memory consumption, so that you can open large files without crashing. File open time should improve across the board as well. This change is not specifically about syntax highlighting though. |
|
@maxbrunsfeld just gave it a try on a 40mb / 1mil line JSON file and it works great. Thank you for your hard work!!! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Problem
A major issue with Atom's current handling of large files is that the
DisplayLayeruses so much memory that v8 can run out of heap space, causing a hard crash.The reason that the
DisplayLayeruses so much memory is that it stores all of the transformations that are applied to the text when it is displayed. These transformations include soft-wraps, folds, hard-tab expansion, atomic soft-tab clipping, indent guides, and invisible-character substitution. They are stored in a data structure called the DisplayIndex, which uses a binary tree to allow content to be accessed efficiently in terms of both buffer or screen coordinates.The Optimization
Translating between arbitrary buffer and screen coordinates is an important responsibility of the DisplayLayer, but many of the transformations listed above do not affect position translation. The only transformations that do affect position translation are soft-wraps, folds, and hard-tabs.
This PR takes advantage of that fact by avoiding storing any of the other transformations, and computing them only as needed, when rendering lines of text. This means that the data stored by the DisplayLayer is now sparse with respect to screen lines; nothing needs to be stored for screen lines without folds, soft-wraps or hard tabs.
We now store the
DisplayLayer's soft-wraps and folds in aPatch- a data structure that we already use elsewhere in this library, for representing groups of changes in the buffer'sHistory, and for aggregating changes to pass toonDidChangeTextcallbacks.As part of this effort, we have rewritten the Patch in C++ and with a more compact representation, in order to further reduce Atom's memory consumption when editing large files.