Byron wrote this poem in 1806 and it was included in the very first edition of Fugitive Pieces.  The "lady" is Julia Leacroft, a girl Byron romanced while living in Southwell.

I have used the version from the Fugitive Pieces, as it was originally written.  You will note that some wording and punctuation differs in later  publications . . . in my opinion,  those changes also changed the meaning in the lines of the poem.