• View
• Edit
• Upload
• History
• Print

InteractiveLyX

1.  Interactive LyX - Status, Plan, Issues

Through some extension in LyX GUI, remote users will be able to connect with each other and concurrently edit the same document.

1.1  Background Information

Basics

In order to have a better understanding go through the following basic structure

• LFUN:

• Buffer:

• BufferView:

• Cursor:

       Cell Index (row, column)
       Paragraph Offset (which paragraph)
       Position (position inside a paragraph)

Historical Information:

A proof-of-concept of this feature has been sketched out as a patch on the LyX mailing list (http://www.lyx.org/trac/ticket/7964), but it is a bad hack, that stands up only a few seconds then it crashes.

Keeping separate cursors for each user in a BufferView will become complex. An ingenious solution to this is to keep separate remote BufferView for each remote user, which will act on the same local Buffer. This way, each remote user will have his separate cursor, and even undo, redo, delete will work locally.

In this patch, each client holds a remote BufferView of the server. Whereas, the server holds remote BufferViews of all the clients

• Initiation:

• Propagation:

• Termination:

• Problems:

1.2  Issues & criticalities

Version Control System

When a LFUN hits the remote BufferView and it is dispatched, it acts on a version of the document that might be slightly different, as modified by the local user concurrently. It is important to clarify what revision of the document each LFUN applies to. So the collaborative editing should be seen making an analogy to a VCS like ‘git’, as continuously committing local changes, pulling and pushing changes across Buffers of the users participating in the collaborative session.

When edits are very close, it should be important to consider what revision each LFUN applies to. A failure scenario can be "User1's and User2's cursors are at the end of a para. Now if U1 starts typing characters, and U2 types 'right' before receiving those LFUNs , U2's cursor will be in next paragraph. where as his remote BufferView's Cursor will still stay in the previous paragraph"

To handle this, each BufferView should maintain a version number and also simultaneously keep track of relative Cursor change, across different version. So the remote BufferView 2 (acting on Buffer 1), will keep track of cursor changes across different versions of Buffer 1. Similarly , remote BV1 will keep track of cursor changes across different versions of B2.

Whenever an LFUN is sent it should be packed with target version and current version it's buffer is in.

Send{ LFUN, target_version, cur_version}

the receiving client will than adjust his cursor according to target_version. Sending cur_version will help in clearing out the unnecessary history up-to cur_version .

Cursor Validity

As discussed previously, a cursor is represented by index, paragraph offset and position. Care should be taken to maintain cursor validity, simple example would be if local cursor is on 4th paragraph, but remote user deletes 2nd paragraph, the local cursor is no longer valid because its paragraph offset is incorrect. This can be handled by tracking cursor after every edit LFUN i.e. Delete, copy, undo etc.

Whenever a user makes changes in his local Buffer, all the remote BufferViews should track this change and keep a history of how the cursor position was affected with each version change. A remote cursor will be effected, only if its position is after the editing cursor.

If a user makes a change in his Buffer, the change in the cursor position in his local BufferView should be tracked to get the difference( for example if he typed a character, then difference in cursor->pos_type will be 1). This information can be used by remote BufferViews to update their cursor (only if their cursor is ahead of the editing cursor)and then update their history queue, by enqueueing (cursor_postion, version). This way, history of revisions can be maintained.

Another important point is to remove the history once it becomes useless.It will only become useless , when it is no longer being referenced by the remote BufferView. Or in other words, user 2's remote BufferView knows, in which latest revision User 1's Buffer is. So, whenever an incoming LFUN reaches remote BufferView , it checks the target_version, and sees that versions less than target_version are no longer referenced and hence they can be de-queued.

Heavy Processing

Suppose 1 of the client puts a higher resolution image, it might create problems as other clients will have to download that image separately. Can be handled if the image is not transferred rather just its dimensions.

There will be many other unforeseen scenarios that will have to be handled correctly.

1.3  Project Status

Achievements so far..

Port existing patch to current trunk

Existing proof-of-concept patch (by Tommaso Cucinotta) was ported successfully to the current trunk.

Patch Link: https://gist.github.com/HotSushi/75ed590070e293f1b563(approve sites)

Server Queue

In order to maintain consistency of a document, the easiest way would be to keep a queue in the server. And whenever a LFUN is generated, instead of dispatching it locally it is sent to the server. The server then enqueues the incoming LFUN and periodically dequeues. The dequeued LFUN is then sent to every client. Because of this, every user will dispatch the LFUNs in same order. Hence the document will be in consistent state.

Problem: Even though this approach is easier, it has a big drawback of speed. Every LFUN in order to get dispatched, will have to travel to the server, wait in its queue, get dequeued, travel back and then get dispatched. The performance will be laggy.

A proof-of-concept of this feature has been sketched out as a patch

Patch Link: https://gist.github.com/HotSushi/f39fde215b3ab9d6fee2(approve sites)

Handling Mouse Click

The above patch worked fine, but there were couple of issues, one of them being relocation of cursor because of mouse click. The receiving end was not understanding when the remote cursor moved, if there was a mouse click. There was a quick fix for that, send LFUN along with the cursor location where it has to be dispatched. I spent majority of my time understanding how the cursor works, and its behavior inside insets(DocIterator with stack of CursorSlice) and how to serialize and de-serialize it.

I was finally able to code a patch which serialized cursor location, sent that data over the network and deserialize that data and relocate the cursor.

Patch Link: https://gist.github.com/HotSushi/5f447ac1ea60d8a9ed78(approve sites)

Video of working: https://www.youtube.com/watch?v=jZbOGGdb-y8

Class to vehicle additional Information

As we will be working with version of documents, and each LFUN will have to know which version it has to be dispatched on, additionally it will also have to carry LFUN as well as cursor location. It would be better if some separate module handled packing and unpacking of data.

A class "LFUNHolder" was designed which provided an api for attaching and detaching data such as version, LFUN, Cursor etc.

Patch Link: https://gist.github.com/HotSushi/8c4229cb572aa9861d44(approve sites)

Introducing Version

After this, I began working with document versions. Each state of the document should be given a different version. I incremented a counter to signify change in version after every edit. During implementing this, we faced a dilemma whether or not to send cursor movement LFUNs across, each having their own advantages and disadvantages. We decided on not sending movement LFUNs,so as to prevent unnecessary data transfer, and cases where window size was a concern (LFUN_HOME, LFUN_END the cursor will end up in 2 different locations across).

Logic

A history table was maintained, <Version_no, LFUN, Cursor_Difference_after_dispatch, Dispatch_location, Client_id> . The incoming LFUN than traverses the table to predict current Cursor Location. In order to handle dispatch at same cursor location across both ends (User1 ,U2 dispatch at same location ), each client was given a priority, and lower priority client moved its cursor according to changes made by higher priority client.

Patch Link: https://gist.github.com/HotSushi/c7e787c75d600559f147(approve sites)

Forcing lag to simulate slow network

In order to test the behavior of LyX in slow network, I designed a LyXFunction 'collaborate-speed', that would slow down speed of receiving data. The working of this LFUN is shown in this video.

Video Link: http://youtu.be/69ZrO95Wl3A(approve sites)

Correct behavior in slow network

After a lot of testing, we were able to arrive at a solution that would handle simple character inserts in a slow network. Here is a video showing different use cases. We upgraded the patch to handle Insets as well, by tracking cursor change across insets. Sanitizing the cursor had to be postponed to final step(just before it was re-adjusted), to prevent vector array-out-of-bound crashes.

Video Link: http://youtu.be/Mh-1OL7I5qE(approve sites)

Patch Link: https://gist.github.com/HotSushi/efc43fde4793f553c231(approve sites)

Sending selection data

When a cursor was sent across, only its 'location' DocIterator was sent. In order for 'selection' to work 'anchor_' DocIterator was also sent, and accordingly the data was reassembled together to give correct results.

Video Link: http://youtu.be/Lrv5pOsB2vQ(approve sites)

1.4  Features

• Simple text edits (Tested )

• Inset edits (Tested )

• Automatic version synchronization (working )

• Forcing Lag (Tested )

• Cursor movements (mouse + keyboard) (working )

• Selection in buffer ( working )

• Copy/Paste (Not Tested )

• Manually Pulling buffer in case of Ambiguity(working )

1.5  What's left

• Integrating XMPP chat
• Supporting more than 2 users
• Undo and other necessary LFUNS

Category: GSoC