metadata
license: apache-2.0
tags:
- summarization
datasets: billsum
widget:
- text: >-
The people of the State of California do enact as follows: SECTIONHEADER
Section 1170.02 is added to the Penal Code, to read: 1170.02. A prisoner
is not eligible for resentence or recall pursuant to subdivision (e) of
Section 1170 if he or she was convicted of first-degree murder if the
victim was a peace officer, as defined in Section 830.1, 830.2, 830.3,
830.31, 830.32, 830.33, 830.34, 830.35, 830.36, 830.37, 830.4, 830.5,
830.6, 830.10, 830.11, or 830.12, who was killed while engaged in the
performance of his or her duties, and the individual knew, or reasonably
should have known, that the victim was a peace officer engaged in the
performance of his or her duties, or the victim was a peace officer or a
former peace officer under any of the above-enumerated sections, and was
intentionally killed in retaliation for the performance of his or her
official duties. SECTIONHEADER Section 3550 of the Penal Code is amended
to read: 3550. Notwithstanding any other law, except as provided in
subdivision (b), if the head physician of an institution in which a
prisoner is incarcerated determines, as provided in this section, that the
prisoner is permanently medically incapacitated with a medical condition
that renders him or her permanently unable to perform activities of basic
daily living, and results in the prisoner requiring 24-hour care, and that
incapacitation did not exist at the time of sentencing, the prisoner shall
be granted medical parole if the Board of Parole Hearings determines that
the conditions under which he or she would be released would not
reasonably pose a threat to public safety. This section does not alter or
diminish the rights conferred under the Victims Bill of Rights Act of 2008
. Subdivision (a) does not apply to any of the following: A prisoner
sentenced to death or life in prison without possibility of parole. A
prisoner who is serving a sentence for which parole, pursuant to
subdivision (a), is prohibited by any initiative statute. A prisoner who
was convicted of first-degree murder if the victim was a peace officer, as
defined in Section 830.1, 830.2, 830.3, 830.31, 830.32, 830.33, 830.34,
830.35, 830.36, 830.37, 830.4, 830.5, 830.6, 830.10, 830.11, or 830.12,
who was killed while engaged in the performance of his or her duties, and
the individual knew, or reasonably should have known, that the victim was
a peace officer engaged in the performance of his or her duties, or the
victim was a peace officer or a former peace officer under any of the
above-enumerated sections, and was intentionally killed in retaliation for
the performance of his or her official duties. When a physician employed
by the Department of Corrections and Rehabilitation who is the primary
care provider for a prisoner identifies a prisoner that he or she believes
meets the medical criteria for medical parole specified in subdivision
(a), the primary care physician shall recommend to the head physician of
the institution where the prisoner is located that the prisoner be
referred to the Board of Parole Hearings for consideration for medical
parole. Within 30 days of receiving that recommendation, if the head
physician of the institution concurs in the recommendation of the primary
care physician, he or she shall refer the matter to the Board of Parole
Hearings using a standardized form and format developed by the department,
and if the head physician of the institution does not concur in the
recommendation, he or she shall provide the primary care physician with a
written explanation of the reasons for denying the referral.
Notwithstanding any other provisions of this section, the prisoner or his
or her family member or designee may independently request consideration
for medical parole by contacting the head physician at the prison or the
department. Within 30 days of receiving the request, the head physician of
the institution shall, in consultation with the prisoners primary care
physician, make a determination regarding whether the prisoner meets the
criteria for medical parole as specified in subdivision (a) and, if the
head physician of the institution determines that the prisoner satisfies
the criteria set forth in subdivision (a), he or she shall refer the
matter to the Board of Parole Hearings using a standardized form and
format developed by the department. If the head physician of the
institution does not concur in the recommendation, he or she shall provide
the prisoner or his or her family member or designee with a written
explanation of the reasons for denying the application. The Department of
Corrections and Rehabilitation shall complete parole plans for inmates
referred to the Board of Parole Hearings for medical parole consideration.
The parole plans shall include, but not be limited to, the inmates plan
for residency and medical care. Notwithstanding any other law, medical
parole hearings shall be conducted by two-person panels consisting of at
least one commissioner. In the event of a tie vote, the matter shall be
referred to the full board for a decision. Medical parole hearings may be
heard in absentia. Upon receiving a recommendation from the head physician
of the institution where a prisoner is located for the prisoner to be
granted medical parole pursuant to subdivision (c) or (d), the board, as
specified in subdivision (f), shall make an independent judgment regarding
whether the conditions under which the inmate would be released pose a
reasonable threat to public safety, and make written findings related
thereto. Notwithstanding any other law, the board or the Division of Adult
Parole Operations shall have the authority to impose any reasonable
conditions on prisoners subject to medical parole supervision pursuant to
subdivision (a), including, but not limited to, the requirement that the
parolee submit to electronic monitoring. As a further condition of medical
parole, pursuant to subdivision (a), the parolee may be required to submit
to an examination by a physician selected by the board for the purpose of
diagnosing the parolees current medical condition. In the event such an
examination takes place, a report of the examination and diagnosis shall
be submitted to the board by the examining physician. If the board
determines, based on that medical examination, that the persons medical
condition has improved to the extent that the person no longer qualifies
for medical parole, the board shall return the person to the custody of
the department. Notwithstanding any other law establishing maximum periods
for parole, a prisoner sentenced to a determinate term who is placed on
medical parole supervision prior to the earliest possible release date and
who remains eligible for medical parole, shall remain on medical parole,
pursuant to subdivision (a), until that earliest possible release date, at
which time the parolee shall commence serving that period of parole
provided by, and under the provisions of, Chapter 8 of Title 1.
Notwithstanding any other law establishing maximum periods for parole, a
prisoner sentenced to an indeterminate term who is placed on medical
parole supervision prior to the prisoners minimum eligible parole date,
and who remains eligible for medical parole, shall remain on medical
parole pursuant to subdivision (a) until that minimum eligible parole
date, at which time the parolee shall be eligible for parole consideration
under all other provisions of Chapter 8 of Title 1. The Department of
Corrections and Rehabilitation shall, at the time a prisoner is placed on
medical parole supervision pursuant to subdivision (a), ensure that the
prisoner has applied for any federal entitlement programs for which the
prisoner is eligible, and has in his or her possession a discharge medical
summary, full medical records, parole medications, and all property
belonging to the prisoner that was under the control of the department.
Any additional records shall be sent to the prisoners forwarding address
after release to health care-related parole supervision. The provisions
for medical parole set forth in this title shall not affect an inmates
eligibility for any other form of parole or release provided by law. (1)
Notwithstanding any other law, the Department of Corrections and
Rehabilitation shall give notice to the county of commitment and the
proposed county of release, if that county is different than the county of
commitment, of any medical parole hearing as described in subdivision (f),
and of any medical parole release as described in subdivision (g). Notice
shall be made at least 30 days, or as soon as feasible, prior to the time
any medical parole hearing or medical parole release is scheduled for an
inmate receiving medical parole consideration, regardless of whether the
inmate is sentenced either determinately or indeterminately.
base_model: led-base-16384
model-index:
- name: d0r1h/LEDBill
results:
- task:
type: summarization
name: Summarization
dataset:
name: billsum
type: billsum
config: default
split: test
metrics:
- type: rouge
value: 38.6502
name: ROUGE-1
verified: true
- type: rouge
value: 18.5458
name: ROUGE-2
verified: true
- type: rouge
value: 25.6561
name: ROUGE-L
verified: true
- type: rouge
value: 33.1575
name: ROUGE-LSUM
verified: true
- type: loss
value: 2.1305277347564697
name: loss
verified: true
- type: gen_len
value: 288.372
name: gen_len
verified: true
Longformer Encoder-Decoder (LED) fine-tuned on Billsum
This model is a fine-tuned version of led-base-16384 on the billsum dataset.
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
How to use
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("d0r1h/LEDBill")
model = AutoModelForSeq2SeqLM.from_pretrained("d0r1h/LEDBill", return_dict_in_generate=True).to(device)
case = "......."
input_ids = tokenizer(case, return_tensors="pt").input_ids.to(device)
global_attention_mask = torch.zeros_like(input_ids)
global_attention_mask[:, 0] = 1
sequences = model.generate(input_ids,
global_attention_mask=global_attention_mask).sequences
summary = tokenizer.batch_decode(sequences,
skip_special_tokens=True)
Evaluation results
When the model is used for summarizing Billsum documents(10 sample), it achieves the following results:
| Model | rouge1-f | rouge1-p | rouge2-f | rouge2-p | rougeL-f | rougeL-p |
|---|---|---|---|---|---|---|
| LEDBill | 34 | 37 | 15 | 16 | 30 | 32 |
| led-base | 2 | 15 | 0 | 0 | 2 | 15 |
This notebook shows how led can effectively be used for downstream task such summarization.