Re: [Gluster-devel] [Gluster-users] PLEASE READ ! We need your opinion. GSOC-2014 and the Gluster community

[Kaushal M]

On Tue, Mar 18, 2014 at 9:08 PM, John Mark Walker <address@hidden> wrote:
> Thanks, Kaushal!
>
> The next steps are the following:
>
> - find out the deadlines for student proposals. If we're going with the 
> Fedora umbrella, we need to get any student proposals submitted very soon
The deadline is this Friday, March 21. We've got 3 days more.

> - write out proposals on fedora project ideas page
This is the ideas page -
http://fedoraproject.org/wiki/Summer_coding_ideas_for_2014
I'm not sure if we should be adding all the ideas I listed earlier
there as some are pretty easy to finish whereas others will take
longer than the GSOC timeline to finish. Also, the ideas need to be in
a specific format.
If I could get help doing this, we could add all.

> - submit proposals, along with proposed mentors
> - track down Vipul and see if he's still interested - I believe he said he 
> was working with KP? KP - can you confirm this?
KP is on vacation and will be back in April, but I can confirm on his
behalf as he had talked to me about this. Since KP isn't around, Vijay
said he'll be reaching out to Vipul.

>
> We need to move quickly if we have any hope of getting projects submitted for 
> this year. Please ask me for any help you need - if you don't know the Fedora 
> folks involved, I can intro you.
>
> Thanks, everyone!
>
> -JM
>
>
> ----- Original Message -----
>> I had a discussion with some developers here in the office regarding
>> this. We created a list of ideas which we thought could be suitable
>> for student projects. I've added these to [1]. But I'm also putting
>> them on here for more visibility.
>>
>> (I've tried to arrange the list in descending order of difficulty as I find
>> it)
>>
>> . Glusterd services high availablity
>>     Glusterd should restart the processes it manages, bricks, nfs
>> server, self-heal daemon & quota daemon, whenever it detects they have
>> died.
>> . glusterfsiostat - Top like utility for glusterfs
>>     These are client side tools which will display stats from the
>> io-stats translator. I'm not currently sure of the difference between
>> the two.
>> . ovirt gui for stats
>>     Have pretty graphs and tables in ovirt for the GlusterFS top and
>> profile commands.
>> . monitoring integrations - munin others.
>>     The more monitoring support we have for GlusterFS the better.
>> . More compression algorithms for compression xlator
>>     The onwire compression translator should be extended to support
>> more compression algorithms. Ideally it should be pluggable.
>> . cinder glusterfs backup driver
>>     Write a driver for cinder, a part of openstack, to allow backup
>> onto GlusterFS volumes
>> . rsockets - sockets for rdma transport
>>     Coding for RDMA using the familiar socket api should lead to a
>> more robust rdma transport
>> . data import tool
>>     Create a tool which will allow already importing already existing
>> data in the brick directories into the gluster volume. This is most
>> likely going to be a special rebalance process.
>> . rebalance improvements
>>     Improve rebalance preformance.
>> . Improve the meta translator
>>     The meta xlator provides a /proc like interface to GlusterFS
>> xlators. We could further improve this and make it a part of the
>> standard volume graph.
>> . geo-rep using rest-api
>>     This might be suitable for geo replication over WAN. Using
>> rsync/ssh over WAN isn't too nice.
>> . quota using underlying fs quota
>>     GlusterFS quota is currently maintained completely in GlusterFSs
>> namespace using xattrs. We could make use of the quota capabilities of
>> the underlying fs (XFS) for better performance.
>> . snapshot pluggability
>>     Snapshot should be able to make use of snapshot support provided
>> by btrfs for example.
>> . compression at rest
>>     Lessons learnt while implementing encryption at rest can be used
>> with the compression at rest.
>> . file-level deduplication
>>     GlusterFS works on files. So why not have dedup at the level files as
>>     well.
>> . composition xlator for small files
>>     Merge smallfiles into a designated large file using our own custom
>> semantics. This can improve our small file performance.
>> . multi master geo-rep
>>     Nothing much to say here. This has been discussed many times.
>>
>> Any comments on this list?
>> ~kaushal
>>
>> [1] http://www.gluster.org/community/documentation/index.php/Projects
>>
>> On Tue, Mar 18, 2014 at 9:07 AM, Lalatendu Mohanty <address@hidden>
>> wrote:
>> > On 03/13/2014 11:49 PM, John Mark Walker wrote:
>> >>
>> >> ----- Original Message -----
>> >>
>> >>> Welcome, Carlos.  I think it's great that you're taking initiative here.
>> >>
>> >> +1 - I love enthusiastic fresh me^H^H^H^H^H^H^H^Hcommunity members! :)
>> >>
>> >>
>> >>> However, it's also important to set proper expectations for what a GSoC
>> >>> intern
>> >>> could reasonably be expected to achieve.  I've seen some amazing stuff
>> >>> out of
>> >>> GSoC, but if we set the bar too high then we end up with incomplete code
>> >>> and
>> >>> the student doesn't learn much except frustration.
>> >>
>> >> This. The reason we haven't really participated in GSoC is not because we
>> >> don't want to - it's because it's exceptionally difficult for a project of
>> >> our scope, but that doesn't mean there aren't any possibilities. As an
>> >> example, last year the Open Source Lab at OSU worked with a student to
>> >> create an integration with Ganeti, which was mostly successful, and I
>> >> think
>> >> work has continued on that project. That's an example of a project with
>> >> the
>> >> right scope.
>> >
>> >
>> > IMO integration projects are ideal fits for GSoc. I can see some
>> > information
>> > in Trello back log i.e. under "Ecosystem Integration". But not sure of
>> > their
>> > current status. I think we should again take look on these and see if
>> > something can be done through GSoc.
>> >
>> >
>> >>>> 3) Accelerator node project. Some storage solutions out there offer an
>> >>>> "accelerator node", which is, in short, a, extra node with a lot of RAM,
>> >>>> eventually fast disks (SSD), and that works like a proxy to the regular
>> >>>> volumes. active chunks of files are moved there, logs (ZIL style) are
>> >>>> recorded on fast media, among other things. There is NO active project
>> >>>> for
>> >>>> this, or trello entry, because it is something I started discussing with
>> >>>> a
>> >>>> few fellows just a couple of days ago. I thought of starting to play
>> >>>> with
>> >>>> RAM disks (tmpfs) as scratch disks, but, since we have an opportunity to
>> >>>> do
>> >>>> something more efficient, or at the very least start it, why not ?
>> >>>
>> >>> Looks like somebody has read the Isilon marketing materials.  ;)
>> >>>
>> >>> A full production-level implementation of this, with cache consistency
>> >>> and
>> >>> so on, would be a major project.  However, a non-consistent prototype
>> >>> good
>> >>> for specific use cases - especially Hadoop, as Jay mentions - would be
>> >>> pretty easy to build.  Having a GlusterFS server (for the real clients)
>> >>> also be a GlusterFS client (to the real cluster) is pretty
>> >>> straightforward.
>> >>> Testing performance would also be a significant component of this, and
>> >>> IMO
>> >>> that's something more developers should learn about early in their
>> >>> careers.
>> >>> I encourage you to keep thinking about how this could be turned into a
>> >>> real
>> >>> GSoC proposal.
>> >>
>> >> Excellent. This has possibilities.
>> >>
>> >> Another possibility is in the mobile app space. I think it would be
>> >> awesome to port GFAPI to Android, for example. Or to make use of the
>> >> python
>> >> or ruby bindings for GFAPI to create a server-side RESTful API that a
>> >> mobile
>> >> app can access.
>> >>
>> >> -JM
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Gluster-users mailing list
>> >> address@hidden
>> >> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>> >
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > address@hidden
>> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>