It took some convincing to get Sean Astin to play the part of Drew Barrymore‘s body-builder brother in 50 First Dates — and it took even more convincing to get him to portray Doug with a lisp.

On this morning’s episode of The Drew Barrymore Show, the beloved talk show host reunited with her onscreen brother to talk about his role in Elf on Broadway. But before they got into his latest career venture, they looked back on working together in the 2004 rom-com.

According to Astin, he had just finished filming the Lord of the Rings trilogy when Adam Sandler approached him about starring in 50 First Dates. He said he had to gain “a million pounds” to play a “stupid fat hobbit” in Lord of the Rings and just when he had lost the weight, he was asked to gain it back for 50 First Dates.

“I lost like 30 pounds and was just skinny and having so much fun on this series,” he said, referring to the Showtime series Jeremiah. “And then Adam said, ‘We’d love you to do this. Can you put on 30 pounds?’ I was like, ‘Oh, my god. Of course, I’ll put on 30 pounds.’”

Astin said production paid for a trainer, noting that he was “eating and lifting weights, which is different from eating and watching TV.”

Barrymore then added that she remembered when Astin created Doug’s iconic voice.

“Oh, yeah. I didn’t want to do that!” he admitted. “My whole career is about me not wanting to do stuff and people telling me, ‘No, you gotta do it.’ And then I do it and it’s like, ‘Oh, that’s good.’”

When director Peter Segal suggested he do a lisp for his audition, Astin — whose wife convinced him to take on the audition in the first place — initially thought it was “cheesy.”

“I recorded myself in front of our little house doing it and then I started to have fun with it,” he said. “I did the lisp and it was much better.”

The Drew Barrymore Show airs on weekdays on CBS. You can check the website for local airtimes.