Add ChunkedUArray

[adinapoli]

This is a wip implementation of ChunkedUArray. Please chime in if you think what I have got so far looks fishy.

Feature list:

• Datatype definition
• Monoid instance functions
• IsList instance functions
• Sequential instance functions
• IndexedCollection instance functions
• Collection instance functions
  • take
  • cons
  • find
  • drop
  • snoc
  • sortBy (mocked)
  • ...
• Core API functions
  • index
  • unsafeIndex
  • read
  • write
  • ...
• Tests (I backed off as i wanted to wait for the testsuite overhaul).

[vincenthz]

Probably should be you, or "Foundation" :)

[vincenthz]

so far it look fine ! One thing that might be a good idea would be to find a better name for this structure; while ArrayUArray describe the structure quite well, it's not a very nice looking name. if anyone have any suggestion to what we could call it :)

[JanGe]

ChunkedArray? UChunks? UBuffers? (although they all sound weird)

[adinapoli]

ChunkedArray is not bad, and I would prefer an "opaque" name rather than using something with the U in it. What about MultiArray?

[NicolasDP]

MultiArray sounds good

[JanGe]

2DArray? From a MultiArray I would expect support for more than two dimensions.

[vincenthz]

I have to agree with @JanGe. MultiArray makes me think of something with dimensions. There's also a good side too so I guess I could warm up to it

[adinapoli]

Good point guys! I don't mind 2DArray, so I guess we could simply go for it and see "how it feels". At the end of the day it's something easy enough to be changed later with relatively no pain 😉

Something we still haven't figured out though is how to handle the "boxed" variant. So far 2DArray is really a "boxed outer array with an unboxed content", whereas we said it would be useful to have something like Array (Array ty), so looks like the name question is gonna croup up again 😁

[ndmitchell]

I would go for ChunkedUArray and ChunkedArray. The semantics is that data is stored in chunks. Anything mentioning array twice, or 2, or multi, conjures up the wrong idea.

[adinapoli]

I'm slowly making progress on this! I have fixed the implementation for take and will try to fix drop early this week 😉

[adinapoli]

I've added the implementation for find and cons.

Only 200 failing tests to go 😂 😂

[vincenthz]

I'ld recommend just doing the basic, and have everything else using x = fromList . x . toList. it's easier to have everything in place this way (tests, benchs), and it's way less daunting this way. You're making good progress regardless !

[adinapoli]

I'ld recommend just doing the basic, and have everything else using x = fromList . x . toList. it's easier to have everything in place this way (tests, benchs), and it's way less daunting this way.

That's actually a good suggestion! I think once I implement drop the rest of the functions are not too daunting if not for the sorting ones, where sorting needs "compaction" anyway, so I think I will stick with the naive fromList . x . toList suggestion for those -- thanks Vincent! 😉

[adinapoli]

I'm definitely still on this (just working on it very sporadically due to life getting in the way). Once I get a correct drop implementation I'm gonna mock out the rest of the Sequential typeclass with the fromList . toList thing so we can at least start getting green builds!

[adinapoli]

Hey @vincenthz ! I have just completed the first pass of the ChunkedUArray data structure, where only a tiny subsets of operations are implemented "natively" (e.g. take, drop, etc), all the others use the fromList . ops . toList trick.

I'm puzzled about why Travis is failing on some builds, do you have any idea here?

Thank you guys!

[JanGe]

That looks like the issue we fixed in #127 and should go away when you rebase your branch on master.

[adinapoli]

Hey @JanGe ! Thanks for chiming in. I already merged upstream/master and tried a new build in my latest commit, but unfortunately Travis keeps failing.

The weird thing is that is failing only for certain environments (and thus not consistently) with vague errors or StackOverflow ones. Unless I did something really silly in my "native" implementations (the one not using the from/toList trick), I'm wondering if what's causing trouble to Travis is simply the sheer amount of tests it needs to run.

How do you guys think is the best approach to troubleshoot this? Use @ndmitchell 's technique to identify space leaks trying to pinpoint if the cause is my code or if is lurking elsewhere?

[vincenthz]

The stack overflow is due to old ghc that doesn't have support the MINIMUM pragma in typeclass, and not implementing the new methods elem, notElem (which happen to recursively depend on each other, busy looping)

[adinapoli]

not implementing the new methods elem, notElem

Ouch, that's embarrassing, @vincenthz ! I totally missed the extra methods for Collection, my bad! I came up with some interim implementations, just for the sake of making Travis happy.

Looks like we have a green build now, many thanks! (at the moment of writing Travis successfully built all the mandatory envs and is completely the last "failable" one). What do you reckon should be the next steps in order to make progress on this? At which point do you feel we should merge this and spin up ancillary tickets to improve the implementation of the stubbed methods?

[vincenthz]

No worries about the collection new methods, there were added whilst you were adding this, so this is somewhat expected.

yes, I think we shouldn't wait for merging this. It's an awesome step already, and well done for completing the initial phase ! We can now incrementally add and improve things.

[adinapoli]

yes, I think we shouldn't wait for merging this. It's an awesome step already, and well done for completing the initial phase !

Thanks man! Feel free to merge whenever that makes sense. 🎉