The Hermit card features a figure representing wisdom and experience. Here, it represents advice to take some time to reflect. The dream may be a message from your unconscious, but you don't necessarily have to let go of your ex. Take some time to understand what is really happening.
Knight of Swords indicates that something can be changed on an emotional level. The knight figure can be interpreted as a warning to act decisively to deal with the situation. It could mean that you should make a final decision and let your ex go.
Knight of Cups shows that your intuition must be taken into account when making a decision regarding this situation. You need to listen to your feelings rather than react on impulse or act solely on what you see. Follow your instincts and consider all aspects before making a final decision.
From the papers it seems that it is important to know all aspects before making a decision about your relationship with your ex. The most likely answer to your question is therefore "No", i.e. you shouldn't let it go without thinking about it first. All elements involved must be considered and you must act on intuition and feelings rather than reacting on impulse.