When is the optimal time to leave for the island?

[Zhuangzi]

In fact the Abandoned Castle quest can be completed without a single fight. I think there are four stacks inside and you can get around all of them without trouble. So you can do that one first if you like.

The only things that are too hard in Darion at first are: Staff of Destruction, Enemen, and the Knight's Tent guards.

EDIT: oh and Magic Range too.