# Code terminates without training while using accelerate

**URL:** <https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598>\
**Category:** 🤗Accelerate\
**Created:** [April 13, 2024, 8:45pm UTC](https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598 "2024-04-13T20:45:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaydeepb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jaydeepb/32/14906_2.png) [@jaydeepb](https://discuss.huggingface.co/u/jaydeepb)\
**Post date:** [April 13, 2024, 8:46pm UTC](https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598/1 "2024-04-13T20:46:00Z")

</div>

Hi everyone. I’m using accelerate and I don’t know why my code terminates right away. There’s probably something wrong as I could easily train models using accelerate before. I’m not getting any errors–it’s just that the code terminates as soon as I execute the notebook launcher cell. Here’s my code:

```auto
def dtr(): 
    from transformers import Trainer, TrainingArguments
    from accelerate import Accelerator
    from transformers import AutoModelForCausalLM
    model = AutoModelForCausalLM.from_pretrained(model_checkpoint)
    #model.to(device)

    model_name = model_checkpoint.split("/")[-1]
    training_args = TrainingArguments(
        f"models/S/100-10%",
        evaluation_strategy = "epoch",
        learning_rate=2e-5,
        weight_decay=0.01,
        push_to_hub=False,
        num_train_epochs=1,
        per_device_train_batch_size=4, 
        per_device_eval_batch_size=4, 
        save_strategy="steps", 
        save_steps = 242, 
    )

    accelerator = Accelerator()
    trainer = accelerator.prepare(Trainer(
        model=model,
        args=training_args,
        train_dataset=lm_datasets["train"],
        eval_dataset=lm_datasets["validation"],
    )) 
    trainer.train() 

from accelerate import notebook_launcher
args = ()
notebook_launcher(dtr, num_processes=4)

```

---

<div class="post-metadata">

**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](https://discuss.huggingface.co/u/muellerzr)\
**Post date:** [April 13, 2024, 9:28pm UTC](https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598/2 "2024-04-13T21:28:57Z")

</div>

> [@jaydeepb](#):
>
> ```auto
> accelerator = Accelerator()
> trainer = accelerator.prepare(Trainer(
> model=model,
> args=training_args,
> train_dataset=lm_datasets["train"],
> eval_dataset=lm_datasets["validation"],
> )) 
> trainer.train() 
> 
> ```

First thing I see, don’t do this. Trainer uses Accelerate so no need to do any of this 🙂

---

<div class="post-metadata">

**Author:** ![jaydeepb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jaydeepb/32/14906_2.png) [@jaydeepb](https://discuss.huggingface.co/u/jaydeepb)\
**Post date:** [April 13, 2024, 9:32pm UTC](https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598/3 "2024-04-13T21:32:04Z")

</div>

I see, thanks! So would it automatically run the model on multiple GPUs?

Also, I was able to figure out the issue. Turns out it was memory error ☹

---

<div class="post-metadata">

**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](https://discuss.huggingface.co/u/muellerzr)\
**Post date:** [April 13, 2024, 10:12pm UTC](https://discuss.huggingface.co/t/code-terminates-without-training-while-using-accelerate/81598/4 "2024-04-13T22:12:44Z")

</div>

> [@jaydeepb](#):
>
> So would it automatically run the model on multiple GPUs?

Correct! 🙂 There’s nothing special needed to do for DDP etc since accelerate is fully integrated (code-wise)