关于转换数据 #9

Activity

1. Tradewindycc

   self-assigned this

   on Mar 3, 2026
2. Tradewindycc

   added

   bugSomething isn't workingSomething isn't working

   on Mar 3, 2026
3. Tradewindycc

   added the

   Bug

   issue type on Mar 3, 2026
4. Tradewindycc commented on Mar 3, 2026

   

   Tradewindycc

   on Mar 3, 2026

   Contributor

   More actions

   Hi — thanks for reporting this. We are working to fix the datasets right now.
   你好，感谢指出这个问题。我们目前正在修复这个数据集。
5. LeiH5180 commented on Mar 5, 2026

   

   LeiH5180

   on Mar 5, 2026

   More actions

   我临时修改data_loader.py中的 class TransformedDataset(Dataset[T_co]): 类，直接跳过错误的数据，可以往后推流程

   class TransformedDataset(Dataset[T_co]):
       _MAX_RETRIES = 50
   
       def __init__(self, dataset: Dataset, transforms: Sequence[_transforms.DataTransformFn]):
           self._dataset = dataset
           self._transform = _transforms.compose(transforms)
   
       def __getitem__(self, index: SupportsIndex) -> T_co:
           for attempt in range(self._MAX_RETRIES):
               try:
                   return self._transform(self._dataset[index])
               except Exception as e:
                   logger.warning(
                       f"[DataLoader] 跳过损坏的数据样本 index={index}: {e}"
                   )
                   # Pick a different random index and retry
                   index = random.randint(0, len(self._dataset) - 1)
           raise RuntimeError(
               f"连续 {self._MAX_RETRIES} 次尝试均失败，无法获取有效数据样本。"
           )
   
       def __len__(self) -> int:
           return len(self._dataset)

   👍React with 👍1Reacted by Checheng Yu