iteratee ByteStringLike conversions

[Conrad Parker]

On 11 November 2011 18:44, John Lato <jwlato at gmail.com> wrote:
> I think I agree with Michael on this.
>
> I suggested generalizing the types used in zoom-cache in the first
> place because of performance concerns with the original types.  In
> terms of an interface, it may be cleaner to use just ByteStrings.
> From my understanding of zoom-cache, a plain ByteString would probably
> be the desired type most of the time anyway.
>
> I guess my question for Conrad is, do you actually use a stream of
> [Word8]?  If not, I would say to stick with ByteStrings.  If you do,
> then it still may make sense to use just ByteStrings and convert with
> mapChunks where necessary.

John,

thanks for the advice. We're not actually using [Word8] (though
originally I did have just [Word8] as the stream type, hence your
earlier advice to generalize). Accordingly I've switched the
zoom-cache interface to just use ByteString:

https://github.com/kfish/zoom-cache/commit/ba075fe7127c21c8a6987014b0b594da39ede252

cheers,

Conrad.

>
> As to the type class, to my knowledge it doesn't exist.  I think
> ByteStrings make sense for C bindings.  Rather than generalizing the
> type of enumInflate, I would prefer explicitly converting the output
> with mapChunks (or more complex functions when necessary).  There's a
> danger of too much generalization.  If all the input and output types
> are polymorphic, then users need to provide type annotations
> everywhere.  I think it's both simpler and more intuitive to have some
> monomorphic types to anchor code and expectations.  In line with this,
> I'm considering (but not completely sold on) changing enumFile*
> functions to generate only ByteString streams in future releases.
>
> John
>
> On Fri, Nov 11, 2011 at 4:48 AM, Michael Baikov <manpacket at gmail.com> wrote:
>> Are you sure that operating on [Word8] where it represents compressed
>> stream makes sence? To get benefits from compression size of data
>> should be quite large and operating on large lists of binary data is a
>> very good way to destroy the performance. So the idea  should be
>> operating on ByteStrings until you need data extracted and extract
>> data into that format you really want. So making it into generic case
>> is stupid, and if user really wants to destroy it's performance - it
>> is easy to achieve using threadDelay or mapStream B.unpack
>>
>> Also enumInflate in iteratee-compress *should* be implemented in terms
>> of Enumeratee instread of Enumerator, because it's idea is to
>> transform data stream from one type to another and not the iteratee
>> itself.
>>
>>
>> On Fri, Nov 11, 2011 at 12:29 PM, Conrad Parker <conrad at metadecks.org> wrote:
>>> Hi,
>>>
>>> I recently converted zoom-cache to use (I.Nullable, LL.ListLike s
>>> Word8) constraints for byte streams. I then added support for
>>> compressed data blocks, using enumInflate from iteratee-compress:
>>>
>>> enumInflate :: MonadIO m => Format -> DecompressParams -> Enumerator
>>> ByteString m a
>>>
>>> In order to use that unmodified, I wanted a function of type (Iteratee
>>> ByteString m a -> Iteratee s m a), and ended up writing a simple
>>> typeclass for ByteStringLike streams:
>>>
>>>
>>> class (I.Nullable s, LL.ListLike s Word8) => ByteStringLike s where
>>>    likeFromByteString :: ByteString -> s
>>>    likeToByteString   :: s -> ByteString
>>>
>>> instance ByteStringLike [Word8] where
>>>    likeFromByteString = B.unpack
>>>    likeToByteString   = B.pack
>>>
>>> instance ByteStringLike ByteString where
>>>    likeFromByteString = id
>>>    likeToByteString   = id
>>>
>>>
>>> allowing the desired conversion function:
>>>
>>>
>>> iterToByteStringLike :: (I.Nullable s, LL.ListLike s Word8,
>>> ByteStringLike s, Monad m)
>>>                     => Iteratee ByteString m a -> Iteratee s m a
>>> iterToByteStringLike = I.joinI . I.mapChunks likeToByteString
>>>
>>>
>>> This all seems a bit circuitous.
>>>
>>> 1) Does a similar typeclass already exist?
>>>
>>> 2) Is there a simpler way to use enumInflate from an arbitrary byte stream?
>>>
>>> 3) It would be nice to just generalize the type of enumInflate, but it
>>> binds to the zlib C library and expects byte buffers, hence the direct
>>> ByteString interface; it would have to do something similar
>>> internally. Would these to/from iteratee conversion functions be
>>> generally useful for the Iteratee library?
>>>
>>> Code for Data.Iteratee.ByteStringLike is here:
>>> https://github.com/kfish/zoom-cache/blob/zlib/Data/Iteratee/ByteStringLike.hs
>>>
>>> cheers,
>>>
>>> Conrad.
>>>
>>> _______________________________________________
>>> Iteratee mailing list
>>> Iteratee at projects.haskell.org
>>> http://projects.haskell.org/cgi-bin/mailman/listinfo/iteratee
>>>
>>
>> _______________________________________________
>> Iteratee mailing list
>> Iteratee at projects.haskell.org
>> http://projects.haskell.org/cgi-bin/mailman/listinfo/iteratee
>>
>
>