Conversation
DOC-182 Share Data Between Sessions
The ProblemCustomer wants to be able to share data between two different sessions. The ImpactThe resolution will expand the use cases customers are able to use Grid for. Below is the community members use case. Use Case:
The reason not to use datastores is that both Session 1 and Session 2 share the same dataset, and only 1 datastore can be attached per session. Therefore, I cannot move between Session 1&2 without being able to hot swap datastores or have multiples EvidenceThis message comes from Paul Szerlip in the grid community channel. Suggested Solution (optional)Neven suggested the following solution: "…technically after grid sync ssh config to Documentation related to his proposed solution can also be added to the datastores section of the documentation. Tracked |
docs/products/sessions/README.md
Outdated
| ``` | ||
|
|
||
| ## Share Data Between Sessions | ||
| At this time the only supported way to share data between sessions is to mount the sessions locally and copy files to the local mounts. Establishing SSH connection to your sessions is a prerequisite to this. See these [steps](https://github.com/gridai/grid-docs/blob/doc-182-share-data-between-sessions/docs/products/sessions/how-to-ssh-into-a-session.md) to set up SSH connection with your sessions. **Be sure to logout of the interactive session before attempting the next step**. After that you can do the following to create your session mounts: |
There was a problem hiding this comment.
@oojo12 when hyperlinking to an internal link, use relative markdown path so you can update the link here to
(./how-to-ssh-into-a-session.md) otherwise the link will take them to the page in Github
There was a problem hiding this comment.
Huh, why do you need to logout of ixsession before this step?
There was a problem hiding this comment.
@essiequoi making the change now
@nmiculinic because running grid session ssh logs you into the session. Attempting to perform a session mount from within the session gives you an aws public key error. See the screenshot. Thus the provided instructions are workaround for that by doing it from your local machine.
There was a problem hiding this comment.
huh....first things first
Attempting to perform a session mount from within the session gives you an aws public key error
This is not AWS public key error. It has nothing to do with AWS. It's SSH public key error, since a valid private key isn't present on the session; you'd have to use ssh-agent forwarding for this perhaps, this would require CLI changes to include agent forwarding. What do you think @rusenask ?
The second error for modprobe fuse is...interesting. Since we're running sessions within a container, can we use a fuse to mount something using it within a non-privileged container?
https://stackoverflow.com/questions/48402218/fuse-inside-docker This is interesting, albeit a bit dated (2018).
This seems to be a more recent find, 2021 https://stackoverflow.com/questions/68709395/how-to-mount-a-fuse-based-filesystem-on-docker-container-running-on-aws but it still gives a lot of permissions to the session container. CC @filintod for any thoughts.
There was a problem hiding this comment.
@nmiculinic @filintod how would you like to proceed? If it is additional content I would ask that you just commit it directly instead of placing in the comment something for me to copy + paste. It will be more efficient that way. @essiequoi says this is important for the bug bash.
There was a problem hiding this comment.
Regarding fuse; currently mounting fusefs within session will fail since fuse device isn't exposed within the container. Though there's:
https://github.com/JasonChenY/fuse-device-plugin
https://github.com/kuberenetes-learning-group/fuse-device-plugin
fuse device plugins for k8s we could investigate and potentially add; allowing users to use fusefs within the sessions (and mount other sessions within original session, to remind we're using sshfs, that is fuse filesystem when doing session mounting)
|
|
||
| # to share data from a session to another session | ||
| cp -R <dir of interest in created session mount dir> <other session mount dir>/shared_data | ||
| ``` |
There was a problem hiding this comment.
@oojo12 I like how you used the concepts of Session A and Session B in your testing. It makes it clear and easy to use the commands. Could you incorporate that here?
There was a problem hiding this comment.
I kinda wanted to stay away from that here and use more general language. @essiequoi if you still would like the concept of Session A Session B here let me know and I will make that change.
There was a problem hiding this comment.
<dir of interest in created session mount dir> is unclear. you could also use the language source and target (or destination to distinguish between the two sessions
|
@oojo12 do you need another review? if so, re-request |
|
@oojo12 is this one still in progress? we'll want to merge before the docs bug bash |
rerequested review from Neven. I am not sure the path forward requested. |
nmiculinic
left a comment
nmiculinic
left a comment
There was a problem hiding this comment.
I'll put this on hold until we support fuse fs within sessions
3 participants
What does this PR do?
Currently, there is a lack of documentation enabling users to share data between sessions. This PR addresses that by adding instructions for doing this to the datastores section of the documentation.