- Supporting sync, sequential async, and concurrent async iteration
- Limiting it to a small number of orthogonal concepts that compose beautifully to solve problems
- Making it fully tree-shakeable
I built it for myself and have (mostly) been its only user as I refined it. I've used it in lots of personal projects and really enjoyed it.
I recently decided it would be nice to spread that enjoyment so I created a documentation website complete with a playground where you can try out the library.
I hope you enjoy using it as much as I do! Looking forward to hearing your thoughts :)
- I'm not sure I know the use case for all of this logical complexity. Is there a specific use-case you had in mind?
Split the array of elements to process. Set up a promise.all against a function that handles sync/async function calls against each chunk and accumulates an array of results. This isn't the end of the world for any of the practical use cases I can think of. The efficiency loss and memory consumption is miniscule even across hundreds of thousands of records. This is as concurrent as anything else - the event loop is still a mandatory part of the processing loop. You can even defer every 1000 records to the event loop. Javascript is syncronous so there is no concurrency without threading. I can't think of an IO or non-IO function that would create the issues described of not yielding. We used to have chunks of 1000000+ records that would freeze the event loop. We solved this with structured yields to the event loop to allow other processes to continue without dropping TCP packets etc.
If the results are objects, you're not even bloating memory too much. 800KB for 100,000 records seems fine for any practical use case.
I like the idea of an async generator to prevent over-pre-calculating the result set before it's needed but I couldn't justify pivoting to a set of new functions called pipe, map, etc that require a few too many logical leaps for your typical ECMAScript developers to understand.
What's the win here? It must be for some abuse of Javascript that should be ported to WASM. Or some other use-case that I've somehow not seen yet.
- This is nicely done.
One function I've written that I frequently use is a generic iterate <T> function which (in JS/TS land) allows you to loop over a T[], Array<T>, ArrayLike<T>, Iterable<T>, AsyncIterable<T>, including generators and async generators.
It is just easier to always be able to write: "for await (const item of iterate(iterable))" in most places, without worrying about the type of the item I am looping over.
I like the reducers and collectors in your library! Going to try it out.
Also, something new I discovered is the Array.fromAsync() method in JS, which is like Array.from() but for async values. I don't think it is available in all browsers/runtimes yet though.
- great library! I just tried it out and compared it with iter-tools
I like Lfi better, great documentation!
Here is the example I use to compare: https://stackblitz.com/edit/stackblitz-starters-ia9ujg6m?fil...
- "lfi is a lazy functional sync, async, and concurrent iteration library for JavaScript and TypeScript"
Looks good (I understand all the word and even the whole sentence).
But what is it good for? ("What's in it for me?")
What are the use cases? How is it difference / how does it improve on / when should I use it instead of / ... lodash, ramda, functional.js, RxJs, etc.?
- Looks useful and powerful. Nice work!
- I don't understand (but am open to be persuaded), isn't iteration in js now trivially easy and actually quite powerful? Especially since we can now put 'await' everywhere? What are the killer use cases for this lib?
- How does this compare to Effect, or Fluture? There have been so many attempts at improving these things in JS that it's hard to keep track.
- Were you aware of IxJS [0]? I've used that to good success over the years. It's relationship to RxJS keeps it familiar.
Any thoughts on what Lfi does better/different than IxJS?
- Just so I understand: Could this library be used for iterating large collections without blocking the main thread? Basically as a replacement for a worker with synchronous iteration?
- Interesting. How is the concurrency achieved? I don’t see any info about concurrency implementation on the website
- Another fp-ts
- Call me an idiot, but nowhere on the home page does it say it's for JavaScript. The code samples looked like JS, but I don't know every language, so I wasn't sure. It's not until I got into the docs and saw "npm" was a sure we were talking about JS...