Pass local cluster more kwargs when using coiled.function local=True

[evamaxfield]

I am using coiled v1.82.0

I have a pipeline which I can kick off locally with the flag --use-coiled and various tasks in the pipeline will run on different AWS VM types depending on what is needed for each one. When I don't provide the --use-coiled flag. The coiled.function is passed local=True however, one of the tasks in the pipeline utilizes a GPU (or at least is very CPU intensive) and when I try to run the pipeline locally with this task added, the spawned LocalCluster spins up a number of workers and multiple threads per worker.

I dug through the installed coiled code on my local machine and found these lines in the function.py file that seem relevant:

if self._local:
    with _set_local_environ(self._environ or {}):
        cluster = dask.distributed.LocalCluster(
            name=self._name,
            threads_per_worker=self._cluster_kwargs["worker_options"]["nthreads"],
        )
        if isinstance(self._cluster_kwargs["n_workers"], (list, tuple)):
            cluster.adapt(
                minimum=self._cluster_kwargs["n_workers"][0],
                maximum=self._cluster_kwargs["n_workers"][1],
            )

I would love to be able to pass kwargs directly to that LocalCluster initialization but in the meantime, is there any way I can set processes=True and other variables directly.

Maybe something like:

@coiled.function(
    # other args ...
    local=True,
    local_cluster_kwargs={"some": "kwargs", "placed": "here"}
)

[dchudz]

Thanks, this seems like a good idea!