When you look at the Wheel of Fortune card, you can see that it represents a time of change and opportunity. It could indicate that even if the current situation is difficult, there is always a chance to change or take a new direction. This can mean that even if the person is negative, there is still a chance to experience something positive.
The reversed Ace of Swords can be seen as a warning. It could mean that your intuition is correct and that you should be careful not to get too wrapped up in this person's negativity. It could be a sign that you need to keep your distance and take the right precautions.
The Eight of Cups reversed may indicate that your circumstances may not be favorable due to the negative person. It could mean that you will have a hard time improving your luck until you are able to get out of this situation.
The Magician on the obverse suggests that you have all the resources necessary to take control of your situation. It might mean that you have to trust your intuition and your abilities to overcome this difficulty and get lucky.
Bottom line, it seems that meeting this negative person will impact your luck, but not necessarily in a negative way. The cards suggest that you have the resources and intuition to overcome this situation and achieve your goals. So, my answer is yes: you can affect your luck by meeting this negative person, but only if you have the intelligence and skills to do so.