言者無意，聽者有心
yán​zhě​wú​yì​tīng​zhě​yǒu​xīn

to take a casual remark to heart