The Text

1826 Constitutional Amendment:
That the people of this state, in their several towns, shall, at their annual election, and in such manner as the legislature shall direct, elect by ballot their justices of the peace; and the justices so elected in any town shall immediately thereafter meet together, and in presence of the supervisor and town clerk of the said town, be divided by lot into four classes of one in each class, and be numbered one, two, three and four; and the office of number one shall expire at the end of the first year, of number two at the end of the second year, of number three at the end of the third year, and of number four at the end of the fourth year, in order that one justice may thereafter be annually elected; and that so much of the seventh section of the fourth article of the constitution of this state as is inconsistent with this amendment, be abrogated.

The full provision includes both the amendment and the text it amends – NY Constitution 1822, Article IV, Section 7:
The governor shall nominate, by message, in writing, and with the consent of the senate, shall appoint, all judicial officers, except justices of the peace, who shall be appointed in manner following, that is to say: The board of supervisors in every county in this state shall, at such times as the legislature may direct, meet together; and they, or a majority of them so assembled, shall nominate so many persons as shall be equal to the number of justices of the peace to be appointed in the several towns in the respective counties. And the judges of the respective county courts, or a majority of them, shall also meet and nominate a like number of persons; and it shall be the duty of the said board of supervisors, and judges of county courts, to compare such nominations, at such time and place as the legislature may direct. And if, on such comparison, the said boards of supervisors and judges of county courts shall agree in their nominations, in all, or in part, they shall file a certificate of the nominations in which they shall agree, in the office of the clerk of the county; and the person or persons named in such certificates shall be justices of the peace. And in case of disagreement in whole, or in part, it shall be the farther duty of the said boards of supervisors, and judges respectively, to transmit their said nominations, so far as they disagree in the same, to the governor, who shall select from the said nominations, and appoint so many justices of the peace as shall be requisite to fill the vacancies.
Every person appointed a justice of the peace shall hold his office for four years, unless removed by the county court, for causes particularly assigned by the judges of the said court. And no justice of the peace shall be removed until he shall have notice of the charges made against him, and an opportunity of being heard in his defense.