In your typical dungeon fantasy game, your attacks have both an attack roll and a damage roll. Why don’t other abilities have this feature?
A quick experiment: All your abilities have a ‘skill’ roll and an ‘effectiveness’ roll. Skill determines breadth of ability, the character’s capacity to take what they know and apply it to the situation at hand. And ‘effectiveness’ is how well that works.
When you roll to test an ability, roll 1d20+ that ability’s skill rating. Target difficulty is 10+the defender’s skill rating. If you succeed, you roll effectiveness and apply that ‘damage’ to the defender’s ‘stress.’
Stress takes the place of ‘hit points.’ When you take enough stress, you gain a condition reflecting what happened to you. When you gain a condition, mark a couple* of your skills. Those skills are at -1 per mark until the condition is removed. 
Difficulty, in general, is abstracted much like hit points for monsters would be in standard dungeon fantasy. A cliff that resists the efforts of the party might have 10 ‘stress’ that must be overcome by applying climbing skills, or engineering, or athletics until the party’s gotten on top. Meanwhile, slipping stones cause abrasions or concussions, falling causes scrapes and bruises and broken bones, etc. A monster, obviously, needs to be defeated- but by the sword, or by demoralization? Or can it simply be fed? You can give the world around the players various ‘skills’ that reflect how approaches might be taken. A hungry owlbear might have Ravenous +3, Ferocious +2, and Massive +1, so it’s really good at eating, fighting, and being hard to move (in that order). You might have a hard time in a battle, but it’s not too hard to trick. A stuck door might have Rusted +2 and Jammed +1. Picking the lock is a pain, but it’s not especially durable do you can probably shatter it if you have an axe or something.
As a rule of thumb, you can probably give non-players an amount of stress equal to their skill totals times their durability, where durability is explicitly there to make the challenge last longer (and therefore have more opportunities for the environment to make its own actions).
Probably there needs to be an explicit rule that if the players fail, then the challenge gets an opportunity to react using its own skills and rolls, with its own chance to cause stress and inflict conditions.
Items exist in this ruleset to tell you things about the fiction- a sword lets you cut, but you can fight without it. Ropes let you climb back up from places. Poles let you poke. An axe lets you cleave. A helmet might protect you from head trauma. Probably it’d be cool to let appropriate gear give you a +1 to your skill- after all, you gotta bring the right tools for the job.
 I don’t think it’s probably necessary to note which stress condition caused which penalty- if you gain a condition that affects your fighting and one that affects your climbing, it’s probably ok for the condition you gained in a fight to recover and remove the penalty to your climbing. The “human” body/psyche/spirit recovers in a variety of interesting ways, after all. Not that this ruleset specifies that you must play a human, of course. Play a cyber-shark vigilante in a post-human Mars colony for all I care.