System.Runtime.Extensions Area Performance Tests

[mellinoe]

This is a tracking issue describing the current performance tests we have in the System.Runtime.Extensions area, and what coverage is missing. The work items below can be completed independently, and they are all of different priority.

When adding performance tests, please adhere to the guidelines listed here:

Existing Coverage:

There is some existing coverage, but it needs to be audited for conformance with our performance test guidelines.

• Path Add Path.cs performance tests. corefx#19367
• Random Add perf tests for Random.cs. corefx#19561
• Environment -- Add performance tests for Environment.cs. corefx#19623)

Missing Coverage:

• Math, MathF
• MemoryStream
• StreamReader / StreamWriter
• BinaryReader / BinaryWriter
• ArrayList
• StringReader / StringWriter
• TextReader / TextWriter
• BitConverter
• Convert

[stephenmichaelf]

@mellinoe Could you put them in priority order and I can give it a shot from the top? Thanks!

[mellinoe]

Hey @stephenmichaelf, I tried to initially put it in "priority order", but it was really just based off of my intuition. @AlexGhiondea Do you have any thoughts on the priority of test cases above? Is there something that should be "the top priority"?

[jlanng]

Is something in the CI process watching for increases in time taken to run each of these tests?

It'd probably be worth adding something to the perf test guidelines that describes how the data is used.

[stephenmichaelf]

@jlanng Agreed!

@mellinoe Can we change the list to checkboxes? The PR referenced above, dotnet/corefx#19367 has added tests for Path.cs. Maybe we can add check marks then refer to the PRs to the right of each item as they are done?

[mellinoe]

@stephenmichaelf Good idea -- done.

[stephenmichaelf]

@mellinoe Latest update:

Random has been merged
Environment is waiting on feedback for running cross platform
I will start on Math and MathF tonight.

[jlanng]

@mellinoe how are the timings of these tests used?

[mellinoe]

@jlanng Sorry for the late response.

Timing data (and other metrics like memory allocated, instructions retired, etc) is collected regularly for all of our tests and archived. We have an internal tool which we can use to analyze trends in this data, and identify when regressions have occurred. Right now we are at the stage where we just need to "get better tests" so that the data and trends we see are more stable and coherent. But that is the long-term strategy.

[KostaVlev]

Hi guys, I am going to grab the ArrayList..?

[mellinoe]

@KostaVlev Sounds good, go ahead! Make sure to check around and make sure nobody has added anything without consulting the list. I don't think anyone has, but it doesn't hurt to double check.

[WinCPP]

Hi @mellinoe may I work on StreamReader / StreamWriter? There are a few for StreamWriter but none for StreamReader.

[KostaVlev]

@WinCPP seems like nobody is working on these... so grab it :)

[ddizh]

@mellinoe could you update the list to see what else left uncovered?