I would also bite the bullet so to speak and take timing cover off, replace the woodruff key with a new one and use a new or decent replacement pulley. Its not a massive job just takes a bit of time but well worth peace of mind!
As said it also holds the timing and balance chain sprockets in place and you dont want them doing anything else but turning with the crank, unless you want to wave goodbye to your engine...... (y)