Use common matrix interface?

[mikera]

This is a very interesting project! I'm very interested in all the symbolic manipulation stuff.

I notice however that you are creating a new vector / matrix implementation.

Some of us have noticed that there are already many matrix implementation (JBLAS/Clatrix, Parallel Colt, javax.vecmath, EJML, Vectorz, and I'm sure many others....). We're trying to build a common matrix API that will support multiple back-end implementations. See:

• https://github.com/mikera/matrix-api
• Common matrix interface? tel/clatrix#7

Can I encourage you to consider building dj.math on top of the common API? This will give benefits all round:

• You will get the ability to plug and play with powerful back-end matrix libraries
• We'll collectively avoid reinventing some wheels (we don't really need yet more matrix libraries!)
• You will hopefully get easier interop with other Clojure libraries that use the common matrix API, e.g. there is interest in the Incanter community to move towards this
• We'll get valuable feedback from field-testing using the API so we can refine it into something great

[bmillare]

Hi Mike,

The main reason I use my own matrix implementation is because all the
operations, from the ground up, need to be defined in terms of generic
operators. In other words, I cannot use clojure.core/+ -, etc. The elements
of my matrices may potentially be symbols or expressions. I noticed you
choose to use protocols for the operations, which is good for dispatching
performance on the first arg, but for my work, many operations may depend
on several factors, arg types, arg metadata like size, etc, I feel
multimethods or some other dispatching method would be more convenient and
manageable. The main way I see our libraries interacting at the moment is I
would define a clojure code emitter that uses your library to do the
computations. I'll still have to look into this more though.

Best,
Brent

On Sun, Jan 6, 2013 at 9:25 PM, Mike Anderson notifications@github.comwrote:

This is a very interesting project! I'm very interested in all the
symbolic manipulation stuff.

I notice however that you are creating a new vector / matrix
implementation.

Some of us have noticed that there are already many matrix implementation
(JBLAS/Clatrix, Parallel Colt, javax.vecmath, EJML, Vectorz, and I'm sure
many others....). We're trying to build a common matrix API that will
support multiple back-end implementations. See:

• https://github.com/mikera/matrix-api
• Common matrix interface? tel/clatrix#7 Common matrix interface? tel/clatrix#7

Can I encourage you to consider building dj.math on top of the common API?
This will give benefits all round:

• You will get the ability to plug and play with powerful back-end
  matrix libraries

• We'll collectively avoid reinventing some wheels (we don't really
  need yet more matrix libraries!)

• You will hopefully get easier interop with other Clojure libraries
  that use the common matrix API, e.g. there is interest in the Incanter
  community to move towards this

• We'll get valuable feedback from field-testing using the API so we
  can refine it into something great

  —
  Reply to this email directly or view it on GitHubhttps://github.com/Use common matrix interface? #1.

Brent Millare
PhD Student
Johns Hopkins University
Department of Biomedical Engineering
3400 N. Charles Street Hackerman Hall 218
Baltimore, MD 21218
bmillar1@jhmi.edu

[mikera]

Hi Brent,

That certainly makes sense - I can see some very customised logic needed for manipulating the symbolic components of matrices.

Would you be able to use a generic n-dimensional array structure for your purposes? It's not on my roadmap at the moment, but there's always an option to implement one - rather like Python's ndarray. It wouldn't limit what you can put in the cells or what operations you could perform. Any use to you?

Apart from that, having an emitter after the symbolic logic is performed definitely seems to be the right point to interface in general. If you get a chance to have a go at this, do let me know - I'm very keen to get feedback on the matrix API with a few different use cases to make sure it is as useful for as many projects as possible.

All the best,

Mike.

[bmillare]

Hi Mike,

I can't currently think of a case for the n-dimensional array (for the
n-part, I have use for the 2-dimensional generic array) but I can imagine
people would. Personally I just needed something that worked and went for
the simplest implementation I could think of and went with nested vectors.
I inherited some of the benefits of immutability so "associng" and
obtaining minors was very clojure idiomatic. For the matrix sizes I was
using that was fine. If you provide some of the basic manipulation
operators on other implementations that make some of the operations more
efficient, I can build my "higher level" operations out of these and I can
see that being very good. (Although I'd probably do that later down the
road as I probably don't have much time to do that type of optimizations).
I agree with your goal for building a consistent API for the operations.
There will always be tradeoffs for the performance of different operations
depending on the implementation and you want to give people options while
providing a consistent interface.

As a take home message, my only real feedback is that it might be worth
providing some protocols on some low level operations that are useful for
building full matrix manipulation algorithms. This will allow defining some
of the matrix operations as just functions and will help minimize how many
protocols future matrix implementers have to extend to, especially if there
implementation is really for only niche performance cases. People should be
able to choose to just implement a basic subset, or for absolute
performance implement the high level operation directly. I'll be able to
provide more specific feedback in the future.

Best,
Brent

On Mon, Jan 7, 2013 at 7:29 PM, Mike Anderson notifications@github.comwrote:

Hi Brent,

That certainly makes sense - I can see some very customised logic needed
for manipulating the symbolic components of matrices.

Would you be able to use a generic n-dimensional array structure for your
purposes? It's not on my roadmap at the moment, but there's always an
option to implement one - rather like Python's ndarray. It wouldn't limit
what you can put in the cells or what operations you could perform. Any use
to you?

Apart from that, having an emitter after the symbolic logic is performed
definitely seems to be the right point to interface in general. If you get
a chance to have a go at this, do let me know - I'm very keen to get
feedback on the matrix API with a few different use cases to make sure it
is as useful for as many projects as possible.

All the best,

Mike.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1#issuecomment-11979142.

Brent Millare
PhD Student
Johns Hopkins University
Department of Biomedical Engineering
3400 N. Charles Street Hackerman Hall 218
Baltimore, MD 21218
bmillar1@jhmi.edu

[mikera]

Thanks Brent - very useful feedback!

Will see what we can do building some of this into the prototype....