Would This Be Plagiarism?

Plagiarism is the theft of intellectual property. If you take the ideas or worse the words of someone else's published work and include it in yours, you've got a case of plagiarism. But here's an interesting twist. Suppose the idea was yours to begin with. Now, I've got a friend and colleague who wrote a paper and in working up the idea with him, I threw out a metaphor that made it into the paper. As academics, we do this sort of thing all the time, kicking stuff around, bouncing ideas, getting responses. My contribution was unattributed per my instructions, but now working on an article for someone else's book, I find the metaphor I came up with appropriate for my own purposes. If I were to include it without attribution to the original article, would it be plagiarism? It is my idea and even my wording, so I am not stealing anything; but it was published in someone else's piece first. I would not do this, but if I were to include the metaphor in its original wording unattributed, would that be plagiarism?