They should be pretty straight. The less you are sticking with this range of motion the more you are isolating the triceps.
So in other words, the more your arm strays away from you, the more auxilliary muscles come into play to perform the kickback and not the tricep.
Let me add that Triceps kickback are an isolation exercise, and better stick to compound exercises for better growth in the tricep such as Skull crushers or my favorite Close Grip Bench Press. If you want a better iso exercise, try cable pushdown.