Skip to content

Workers / Slaves numbers for IPO on clusters #63

Description

@ThomasPouplin

Sorry to ask this question but I'm a bit lost in parallel computing ...

Could somebody explain the ratio between workers and slaves for running IPO optimally ?

I have access to clusters (>200 CPUs) and would like to optimise the run time on my lipidomics datasets (POS and NEG).

Let's say I use 6 QCs samples to perform the optimization.

I understood the following formula:
Cores = IPO-slaves * XCMS slaves

Where I'm really lost, is where are these workers/slaves described between the Snowparam (n workers) and the optimizeXcmsSet (n slaves).

How can I, for example, set one slave per sample (so 6 slaves), and split workers for each slaves (say 4 workers per slave) ?

At the moment, I just set 8 workers and 8 salves - IPO runs without any errors but maybe not optimally.

This is a part of the code I'm using (on 8 CPUs):

snowparam <- SnowParam(workers = 8, type = "SOCK")
peakpickingParameters = getDefaultXcmsSetStartingParams('centWave') # get defaults

resultPeakpicking = optimizeXcmsSet(
params = peakpickingParameters,
isotopeIdentification = "IPO",
nSlaves = 8,
subdir = 'CentWave'
)

Thanks for your feedback !

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions