Gervinho put Parma ahead with a solo goal in the 15th minute and, after a subdued display, Inter suddenly burst into life when defender De Vrij headed an equaliser and substitute Bastoni grabbed the winner three minutes later.
Inter had reserve goalkeeper Tommaso Berni shown the red card and ordered from the bench for dissent in the second half -- his second dismissal of a season in which he has yet to make an appearance on the pitch.
Inter coach Antonio Conte was suspended for accumulation of yellow cards and his side, held to a 3-3 draw at home by Sassuolo on Wednesday, seemed to miss the ranting of their fiery manager on the touchline.
They fell behind when Gervinho got to the byline, cut back inside to leave one defender on the ground then drove a low shot past Samir Handanovic.
Inter pushed Parma back after the break but the home defence, expertly marshalled by 38-year-old Bruno Alves, rarely looked like being breached except when Alexis Sanchez forced a rare save from Luigi Sepe.
But six minutes from time, Lautaro Martinez headed a corner back into the area and De Vrij flicked it past Luigi Sepe to level.