Fallout 3 Did i make a big mistake?

I was at little lamplight talking to Mayor Macreedy and used the Speech challenge and got it right so he gave me to of the cave fungus and i left and didn't come back for days of game play. One day i decided i would go back and buy some, but the chef said that he heard what i had said and now he wont let me look at his stock. then i discovered andale and saw that they were killers but they had tons of strange meat.can i still trade for fungus?