The cottage actually used to be 2 blocks south, on Kingsbridge & Valentine, and was moved for preservation:
Poe Park Monuments - Edgar Allan Poe : NYC Parks
Here are some details about Poe's time at the cottage, from an 1877 biography, including the death of his wife at age 25. It also says that he had a lot of financial problems which must have been a reason for his moves. The whole book is available at the link
The Life and Poems of Edgar Allan Poe: (A New Memoir by E. L. Didier). And ... - Edgar Allan Poe, Eugene Lemoine Didier - Google Books
p. 96 LIFE OF EDGAR A. POE.
As the Spring of 1846 advanced, the health of Mrs. Poe
declined, and fearing the effect of the prostrating summer
heat of the city upon the feeble health of the lovely and
loved invalid, it was determined to remove
to the country. The pretty little village of Fordham was
chosen for the home of the delicate wife. A tiny Dutch
cottage was rented. It was on the top of a picturesque
hill, a pretty, romantic spot; the antiquated little house
was half buried in fruit trees. This new home was small
enough, only boasting four rooms, two below and two
above ; but it was cool, quiet, and away from the noise
and vexations of New York. The parlor was used by
Poe as a study. Here he wrote " Ulalume," "Eureka,"
and other productions of his "lonesome latter years."
This room was furnished with exquisite neatness and sim
plicity. The floor was laid with red and white matting ;
four cane-seat chairs, a light table, a set of hanging book
shelves, and two or three fine engravings, completed the
furniture.
A gentleman who visited Poe at Fordham, in 1846,
says: "The cottage had an air of taste and gentility that
must have been lent to it by the presence of its inmates.
So neat, so poor, so unfurnished, and yet so charming a
dwelling I never saw. There was an acre or two of
greensward fenced in about the house, as smooth as velvet,
and as clean as the best kept carpet. Mr. Poe was so hand
some, so impassive in his wonderful, intellectual beauty,
so proud and reserved, so entirely a gentleman upon all
occasions so good a talker was he that he impressed
himself and his wishes even without words upon those
with whom he spoke, His voice was melody itself. He
LIFE OF EDGAR A. POE. 97
always spoke low, even in a violent discussion, compel
ling his hearers to listen if they would know his opinion,
his facts, fancies, or philosophy. Mrs. Poe looked very
young ; she had large black eyes, and a pearly whiteness
of complexion, which was a perfect pallor. Her pale
face, her brilliant eyes, and her raven hair, gave her an
unearthly look. One felt that she was almost a disrobed
spirit, and when she coughed it was made certain that she
was rapidly passing away."
As the winter of 1846-7 approached, the affairs of the
little Fordham household grew desperate. The sickness
of his wife and his own ill health at this time incapaci-
tated Poe from literary work, his only source of revenue,
and, consequently, the family were reduced to the last
extremity, wanting even the barest necessaries of life at
a time, too, when Mrs. Poe required the little delicacies
so grateful to the sick. It was at this time that N. P.
Willis made, in The Home Journal, his generous appeal
in behalf of his friend and brother poet. In the course
of his article Mr. Willis said : " Here is one of the finest
scholars, one of the most original men of genius, and one
of the most industrious of the literary profession of our
country ; whose temporary suspension of labor, from
bodily illness, drops him immediately to a level with the
common objects of public charity. There is no interme-
diate stopping-place, no respectful shelter, where, with the
delicacy due to genius and culture, he might secure aid,
98 LIFE OF EDGAR A. POE.
till, with returning health, he could resume his labors,
and his unmortified sense of independence." This article
was gratefully acknowledged by Poe, in a letter dated De
cember 30, 1846. in which, afteralluding to Willis s "kind
and manly comments in The Home Journal" he says:
"That my wife is ill is true, and you may imagine with
what feeling I add that this illness, hopeless from the first,
has been heightened and precipitated by her reception, at
two different periods, of anonymous letters. That I my
self have been long and dangerously ill, and that my ill
ness has been a well-understood thing among my brethren
of the press, the best evidence is afforded by the innumer
able paragraphs of personal and literary abuse with which
I have been lately assailed. This matter, however, will
remedy itself. At the very blush of my new prosperity the
gentlemen who toadied me in the old will recollect them
selves and toady me again. That I am without friends/
is a gross calumny, which I am sure you never could have
believed, and which a thousand noble-hearted men would
have good right never to forgive, for permitting to pass
unnoticed and undenied. I am getting better, and may
add, if it is any comfort to my enemies, that I have little
fear of getting worse. The truth is, I have a great deal to
do, and I have made up my mind not to die till it is done. "
Exactly one month from the date of this letter, that is,
on the 3Oth of January, 1847, the loved wife died. Her
death-bed was the witness of a scene as sad and pathetic
LIFE Of EDGAR A. POE. 99
as ever told by poet or romance writer. The weather was
cold, and Mrs. Poe suffered also from the chills that fol
low the hectic fever of consumption. The bed was of
straw, and was covered only with a spread and sheets, no
blanket. Here the dying lady lay, wrapped in her hus
band s overcoat, with a large tortoise-shell cat in her
bosom. The coat and the cat afforded the only warmth
to the sufferer, except that imparted by her mother chafing
her feet and her husband her hands. And thus died, at
the early age of twenty-five, the wife of America s greatest
genius.